Remote Jobs RockRemote Jobs Rock

Quality Evaluator – Fluent English (Brazil)

📅 Oct 16
English FluencyLarge Language Models (llms)AI Response EvaluationAnalytical Judgment
Apply Now

📜 Description

  • Interact with AI models and tools across a wide range of prompts and topics.
  • Compare AI-generated responses side by side to determine which best meets user needs.
  • Assess accuracy, helpfulness, instruction-following, clarity, and overall quality.
  • Follow detailed rating guidelines and multi-step evaluation workflows.
  • Write clear, well-reasoned explanations and apply evaluation criteria consistently.

🛠️ Requirements

  • Fluency in English, with strong reading comprehension and writing skills.
  • Extensive hands-on experience using LLMs and critically assessing their outputs.
  • Strong analytical judgment and the ability to weigh multiple dimensions of response quality.
  • Excellent attention to detail and the ability to follow complex instructions carefully.
  • Ability to explain evaluation decisions clearly and convincingly.
  • Comfort switching between AI models, tools, and multi-step workflows.
  • Previous experience in AI response evaluation, data annotation, or quality assurance.
  • Experience in editing, research, fact-checking, or analytical writing.
  • Familiarity with evaluation rubrics and side-by-side comparison tasks.
Full job description

Quality Evaluator – English

Help shape the future of AI by evaluating and comparing responses from advanced AI models.

  • Location: Remote

  • Type: Independent contractor / Part-time

  • Commitment: up to 3 months

  • Compensation: $7-$12/hour

About the role

At Anyone AI, we create high-quality training and evaluation data for frontier AI models.

We’re looking for English-fluent experts with strong writing skills, critical thinking, and extensive hands-on experience using large language models (LLMs).

In this role, you’ll interact directly with AI models and compare their responses side by side to determine which best meets the user’s needs. Your evaluations and written explanations will help improve the accuracy, helpfulness, and reliability of AI systems.

What you’ll do

  • Interact with different AI models and tools across a wide range of prompts and topics.

  • Compare two or more AI-generated responses side by side.

  • Assess accuracy, helpfulness, instruction-following, clarity, and overall response quality.

  • Follow detailed rating guidelines and multi-step evaluation workflows.

  • Write clear, well-reasoned explanations supporting your evaluation decisions.

  • Apply evaluation criteria consistently across tasks.

What we’re looking for

  • Fluency in English, with strong reading comprehension and writing skills.

  • Extensive hands-on experience using LLMs and critically assessing their outputs.

  • Strong analytical judgment and the ability to weigh multiple dimensions of response quality.

  • Excellent attention to detail and the ability to follow complex instructions carefully.

  • Ability to explain evaluation decisions clearly and convincingly.

  • Comfort switching between AI models, tools, and multi-step workflows.

Nice to have

  • Previous experience in AI response evaluation, data annotation, or quality assurance.

  • Experience in editing, research, fact-checking, or analytical writing.

  • Familiarity with evaluation rubrics and side-by-side comparison tasks.

SD Solutions

SafetyTech Client #1 | Adversarial Task Writer for AI Security RL Gyms

🕒 5 days ago
SD Solutions👥 201 - 500 employees🏢 Staffing And Recruiting

Design YAML-based adversarial prompt injection tasks for an AI security evaluation platform, test them against frontier models, and validate scenarios that reveal whether agents can be manipulated into violating safety policies.

Prompt InjectionAdversarial TestingYAMLDocker
Mistral

AI Scientist, Agentic CAD (AI4Engineering)

🕒 4 days ago
Mistral👥 501 - 1000 employees🏢 Internet🧠 AI-native

Build training data, verifiers, benchmarks, and agent architectures that help Mistral’s language models reliably plan and execute complex engineering workflows across CAD, CAE, and semiconductor/EDA.

Machine LearningLLM DevelopmentPythonLinux
Adoc-Tm

Surgical AI Scientist

🕒 9 days ago
Adoc-Tm👥 11 - 50 employees🏢 Human Resources

Develop and deploy deep learning models for live surgical video and preoperative medical images, turning clinical data into real-time perceptual intelligence in collaboration with software, algorithm, and clinical teams.

📍 🇨🇦 Montreal - Hybrid💼 Full-Time🎹 Mid-level🧠 AI Research Scientist📢 🇫🇷 French Required📢 🇬🇧 English Required
PythonPytorchComputer VisionDeep Learning

Trusted by Remote Workers