This role is for one of our clients
Compensation: $50 - $200 per hour
A leading AI research organization is seeking advanced LLM power users to evaluate how well AI systems handle personalized, real-world life tasks.
This role is for people who use AI tools heavily in their personal lives and can clearly judge whether an AI response is useful, personalized, realistic, and successful.
Requirements
Who We’re Looking For
Strong candidates have:
Why This Work Matters
LLMs are quickly becoming personal assistants for everyday decisions, but truly useful AI needs to do more than produce generic advice. It needs to understand context, preferences, constraints, tradeoffs, and what success looks like in real life. Your evaluations will help improve how AI systems support people with practical, high-context tasks across food, health, productivity, careers, and learning. This work directly contributes to making AI assistants more personalized, trustworthy, and useful for real-world personal workflows.
Engagement Details
We consider all qualified applicants without regard to legally protected characteristics and provide reasonable accommodations upon request.
Design YAML-based adversarial prompt injection scenarios, test them against frontier models, and validate successful tasks that assess whether AI agents can be manipulated into violating safety policies.
Evaluate AI-generated finance, audit, insurance, compliance, and legal content by reviewing its accuracy and reasoning, identifying gaps, and providing structured feedback to improve model performance.
Evaluate AI-generated finance, audit, insurance, compliance, and legal content by reviewing its accuracy and reasoning, identifying gaps, and providing structured feedback to improve model performance.
Design, build, and deploy production-grade AI agents that automate complex business workflows, owning agent architecture, knowledge systems, evaluation, and cross-functional delivery across enterprise functions.
Evaluate AI conversations for whether they gather decision-relevant context, identify missing information and useful follow-up questions, and provide structured feedback to improve conversational quality.
