Data Analyst (AI Evaluation)

Company Description

Job Description

• Job Title: Data Analyst (AI Evaluation)
• Job Type: Contractor
• Location: Remote

About the hiring company:

Our client is a rapidly growing, venture-backed AI company helping shape the next generation of intelligent systems. By combining world-class human expertise with advanced machine learning workflows, they enable leading AI organizations to build, evaluate, and improve cutting-edge models used across a wide range of industries.

The company works with highly accomplished professionals in fields such as software engineering, finance, healthcare, legal, operations, research, and other specialized domains. These experts contribute directly to the development of advanced AI systems by providing real-world knowledge, evaluations, feedback, and domain-specific judgment that help models reason more accurately and perform more effectively.

Leveraging a proprietary AI-driven talent assessment and matching platform, the organization identifies exceptional professionals globally and connects them with high-impact projects at the forefront of artificial intelligence.

Backed by more than $40 million in funding and supported by a rapidly expanding international network of experts, the company is building critical human intelligence infrastructure for the AI economy and creating meaningful opportunities for professionals to apply their expertise in entirely new ways.

Job Summary:

Engaging Data Analysts to contribute their knowledge and expertise to a customer’s data quality project. In this role, you'll apply your expertise to help train next-generation AI systems. Your work will shape how models learn, reason, and perform through high-quality, real-world input. No prior experience in AI is required — your domain knowledge is what matters.

Scope of Work

1. Review datasets and detailed task outputs for accuracy, completeness, and adherence to guidelines.
2. Identify and flag inconsistencies, errors, and outputs that deviate from provided instructions.
3. Apply structured rubrics and evaluation criteria consistently across large data sets.
4. Exercise sound judgment and document rationale in ambiguous scenarios where guidelines are unclear.
5. Escalate patterns or recurring data issues for further analysis rather than addressing them individually.
6. Provide clear, actionable written and verbal feedback pinpointing issues and suggesting resolution paths.
7. Maintain meticulous attention to detail while working through high volumes of repetitive data review assignments.

Preferred Qualifications

1. 3–5+ years of experience in data review, data quality, quality assurance, or analytical roles.
2. Proven expertise in reviewing, validating, and critiquing data or deliverables based on explicit criteria.
3. Exceptional ability to follow detailed written instructions and apply complex evaluation rubrics precisely.
4. Demonstrated comfort with ambiguity, including making justifiable decisions when guidelines are incomplete.
5. Advanced attention to detail and error-spotting capabilities across repetitive or high-volume work.
6. Strong written and verbal communication skills for documenting findings and providing feedback.
7. Baseline fluency with spreadsheets (sorting, filtering, simple formulas); SQL or hands-on data querying is a plus.

Qualifications

Additional Information

A very attractive and competitive package is offered.