Evaluate AI-generated data science and quantitative solutions, create prompts, datasets, and reference answers, and provide rigorous feedback to improve model accuracy and reliability.
Data Scientist Talent Network
📜 Description
- Design task-specific grading criteria for data science deliverables, including analyses, models, pipelines, experiments, and technical reports.
- Evaluate AI-generated or human-created work against established criteria.
- Write detailed justifications for evaluations and scores.
- Apply consistent, evidence-based judgment to make assessments reproducible and defensible.
- Incorporate senior reviewer feedback and iterate on submitted work.
🛠️ Requirements
- Evaluating AI-generated or human-created work against established criteria
- Providing detailed written justifications for evaluations and scores
- Applying consistent, evidence-based judgment so that assessments are reproducible and defensible
- Incorporating structured feedback from senior reviewers and iterating on submitted work
- 1+ years of professional data science experience
- Experience at a leading technology, research, or quantitative firm (such as top FAANG, AI labs, top-tier quant funds, or equivalent)
- Strong command of Python, SQL, statistical modeling, machine learning, experimentation and causal inference, and translating messy real-world data into rigorous analyses
- Exceptional written communication skills, including the ability to convey technical findings clearly
- A detail-oriented and consistent approach to evaluating complex work
- Comfort receiving feedback and calibrating judgment against established standards
Full job description
This role is for one of our clients
Compensation: $100-$150 per hour
We are building a network of experienced data scientists for potential future projects with leading AI research organizations. These projects may focus on evaluating how effectively AI systems perform real-world data science work.
There is no immediate project opening, but qualified applicants may be contacted as relevant opportunities become available.
Requirements
Potential Responsibilities
Future projects may involve:
- Designing precise, task-specific grading criteria for data science deliverables, including exploratory data analyses, statistical modeling work, machine learning pipelines, experimentation and A/B test write-ups, feature engineering, and technical reports or notebooks
- Evaluating AI-generated or human-created work against established criteria
- Providing detailed written justifications for evaluations and scores
- Applying consistent, evidence-based judgment so that assessments are reproducible and defensible
- Incorporating structured feedback from senior reviewers and iterating on submitted work
Specific responsibilities will vary depending on the project.
Ideal Qualifications
- 1+ years of professional data science experience
- Experience at a leading technology, research, or quantitative firm (such as top FAANG, AI labs, top-tier quant funds, or equivalent)
- Strong command of Python, SQL, statistical modeling, machine learning, experimentation and causal inference, and translating messy real-world data into rigorous analyses
- Exceptional written communication skills, including the ability to convey technical findings clearly
- A detail-oriented and consistent approach to evaluating complex work
- Comfort receiving feedback and calibrating judgment against established standards
We consider all qualified applicants without regard to legally protected characteristics and provide reasonable accommodations upon request.
Similar jobs
Search more Data Scientist jobsBuild and maintain statistical, predictive, and causal analytics for local government datasets, turning community trends into clear insights while automating data workflows and supporting client projects.
Build and evaluate predictive lending models, analyze consumer risk and portfolio trends, and partner with engineering and business teams to improve underwriting strategies, model monitoring, and data-driven decisions.
Optimize payment-risk decisioning by analyzing rules, thresholds, segments, and review strategies, then turning experiments and data into recommendations that reduce unnecessary reviews while maintaining loss performance.
The Data Scientist integrates customer data and develops, validates, and communicates simulation models, visualizations, and recommendations that support healthcare decisions and measurable customer impact.
