🕒 3 days ago
Design and scale AI-powered go-to-market workflows, integrations, and internal tools across sales, marketing, and Customer Success to improve efficiency, intelligence, customer experience, and cost to serve.
Evaluations Team Lead
🕒 7 days ago
AI Model EvaluationBenchmarkingStatistical ValidationData Quality
📜 Description
- Build and maintain a shared evaluation platform for benchmarking and investigating model performance.
- Develop and improve benchmark suites and datasets to assess predictive quality across various business use cases.
- Establish evaluation standards covering statistical metrics, validation strategies, and data quality.
- Design reproducible evaluation pipelines with version-controlled inputs and automated regression checks.
- Evaluate predictive performance, inference latency, and cost efficiency across deployment configurations.
- Support customer proof-of-concept evaluations with reliable tooling and data-driven analysis.
🛠️ Requirements
- Proven experience owning the evaluation of tabular machine learning systems used in production environments or high-impact customer decision-making.
- Strong statistical expertise, including selecting appropriate metrics and validation schemes, estimating uncertainty, comparing models across datasets, and accounting for repeated experimentation.
- Practical experience identifying and preventing data leakage involving preprocessing, feature engineering, joins, temporal dependencies, and related entities across training and evaluation splits.
- Strong proficiency in Python and SQL, with familiarity with scikit-learn, gradient-boosted decision trees, and the development of reliable ML tools or platforms used by multiple teams.
- Demonstrated experience designing fair and reproducible model comparisons, including hyperparameter optimization, resource allocation, and end-to-end inference latency measurement.
- Previous people management experience, including hiring, technical mentoring, team development, and performance management, combined with a willingness to remain actively involved in technical work.
- Excellent written and verbal communication skills, with the ability to collaborate effectively with researchers, software engineers, and customer data scientists.
- A strong analytical mindset and the confidence to challenge unsupported conclusions, communicate uncertainty, and ensure that technical decisions are grounded in reliable evidence.
- Experience evaluating tabular foundation models or automated machine learning (AutoML) systems is an advantage.
- Familiarity with measuring the impact of model optimizations on predictive performance and inference efficiency is desirable.
✨ Benefits
- Competitive compensation package combining salary and equity.
- Comprehensive health coverage for employees and their dependents.
- Paid parental leave for all new parents, including adoptive and surrogacy journeys.
- Relocation assistance for employees moving to join the team at one of the company's office locations.
- Remote working opportunities across eligible locations in Europe and Israel.
- The opportunity to establish evaluation standards and build a new function from the ground up
- A mission-driven, collaborative, and low-ego culture that values diverse perspectives, individual ownership, and proactive decision-making.
- The opportunity to work on advanced AI research and help improve the reliability, predictive performance
Full job description
This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for an Evaluations Team Lead based in Romania.
As an Evaluations Team Lead, you will shape how advanced AI models are evaluated, validated, and improved in production environments. You will lead the development of a shared evaluation platform that enables research, engineering, and customer-facing teams to measure model performance with confidence. Your work will establish rigorous standards for benchmarking, statistical validation, data quality, and reproducibility. Combining hands-on technical leadership with people management, you will build a high-performing team while remaining closely involved in code, experiments, and methodology. You will assess predictive quality, inference latency, and cost efficiency to guide research priorities and product releases. This is an opportunity to define evaluation practices from the ground up and directly influence the reliability and impact of next-generation enterprise AI.
Accountabilities:
Requirements:
Benefits:
Similar jobs
Search more AI Engineer jobs
🕒 3 days ago
Design and own the evaluation, testing, and observability infrastructure for AI agents, building automated pipelines and statistical, adversarial, and regression tests to improve reliability, safety, and performance.
🕒 3 days ago
Design and own the evaluation, testing, and observability infrastructure for AI agents, building automated pipelines and statistical, adversarial, and regression tests to improve reliability, safety, and performance.
🕒 3 days ago
Design and own the evaluation, testing, and observability infrastructure for AI agents, building automated pipelines and statistical, adversarial, and regression tests to improve reliability, safety, and performance.
🕒 3 days ago
Design and own automated evaluation, testing, and observability infrastructure for AI agents, measuring reliability, safety, performance, and security across complex interactions and deployment workflows.