🔥 21 hours ago
As an AI Engineer, you will build AI agents and solutions that enhance engineering workflows, automate tasks, and improve team efficiency.
Member of Technical Staff, Agents
🔥 18 hours ago
AI AgentsMachine LearningData ScienceAgent Evaluation
📜 Description
- Build agents that complete delegated work end to end with minimal user oversight.
- Define quality measures and evaluate agent performance across representative tasks.
- Develop capabilities for existing tools and verify readiness before launch.
- Investigate production failures and deliver measurable, durable fixes.
- Build systems to detect failure patterns and improve recovery when tools or websites fail.
- Create repeatable evaluation practices to assess whether changes improve outcomes.
🛠️ Requirements
- At least 3 years in data science or machine learning, building or delivering evaluation systems, metrics, or product quality measurement.
- Strong production proficiency in Python and SQL, including analysis of large datasets and automated pipelines.
- Experience designing evaluation frameworks with measurable criteria, representative datasets, and appropriate metrics.
- Experience building or operating agents for real users and improving their reliability.
- Familiarity with statistics, experiment design, and realistic end-to-end simulations.
- Experience owning systems through production launch and measuring their impact.
- Experience with post-training, fine-tuning, or reinforcement learning is relevant.
- Working knowledge of Django, React, and TypeScript is useful.
- Typically suited to candidates with fewer than 5 years of engineering experience and a track record of progression or startup ownership.
✨ Benefits
- Base salary: $200,000 to $300,000 annually. Visa sponsorship is not available.
Full job description
About the Role
Join a small, senior team building autonomous AI agents that handle real executive workflows. You will design new capabilities, measure their reliability, and improve them using evidence from real conversations before changes reach users.
What You'll Do
Build agents that complete delegated work end to end with minimal user oversight.
Define trustworthy measures of agent quality and evaluate performance across representative tasks.
Develop new agent capabilities for tools people already use, and verify they are ready before launch.
Investigate production failures and deliver durable, measurable fixes.
Build systems to detect known failure patterns and less predictable issues at scale.
Improve agent recovery when tools or websites fail.
Create repeatable evaluation practices so engineers can assess whether agent changes improve outcomes.
What We're Looking For
At least 3 years in data science or machine learning, building or delivering evaluation systems, metrics, or product quality measurement.
Strong production proficiency in Python and SQL, including analysis of large datasets and automated pipelines.
Experience designing evaluation frameworks with measurable criteria, representative datasets, and appropriate metrics.
Experience building or operating agents for real users and improving their reliability.
Familiarity with statistics, experiment design, and realistic end-to-end simulations.
Experience owning systems through production launch and measuring their impact.
Experience with post-training, fine-tuning, or reinforcement learning is relevant.
Working knowledge of Django, React, and TypeScript is useful.
Typically suited to candidates with fewer than 5 years of engineering experience and a track record of progression or startup ownership.
Compensation & Benefits
Base salary: $200,000 to $300,000 annually. Visa sponsorship is not available.
Location
On-site in Palo Alto, California, with four days per week in the office.
Similar jobs
Search more AI Engineer jobs
🕒 2 days ago
Design, build, and deploy AI systems for enterprise customers, taking solutions from opportunity selection and prototyping through evaluation, production launch, and scale while driving measurable outcomes.
Senior Applied AI Product Engineer (Remote Opportunity)
🕒 yesterday
VetsEZ👥 201 - 500 employees🏢 Information Technology And Services
🕒 yesterday
As an Applied AI Product Engineer, you will design and implement AI-powered features to enhance patient engagement and operational efficiency in a remote Agile environment.
🕒 yesterday
Desenvolve soluções backend em Python para um agente de atendimento com IA, integrando LLMs a sistemas externos e aprimorando APIs, fluxos conversacionais e pipelines de RAG para jornadas B2C e B2B.
🕒 2 days ago
Design, build, and optimize AI-powered workflows that reduce manual work and improve business efficiency, integrating automation tools, APIs, chatbots, and AI assistants across teams.