Remote Jobs RockRemote Jobs Rock

Multilingual Audio Evaluator

📅 Oct 12
📍 🇦🇷 Argentina - Remote🎹 Mid-level🎲 Other📢 🇳🇱 Dutch Required📢 🇮🇹 Italian Required💼 Full-Time
Multilingual Spoken-language ProficiencyJSON Validation And TroubleshootingAudio EvaluationTranscript Labeling
Apply Now

📜 Description

  • Prepare and confirm tools and access before live sessions.
  • Conduct real-time roleplay scenarios with conversational AI agents.
  • Capture recordings, session IDs, and tool-call or console logs.
  • Label transcripts according to project guidelines and score interactions using structured rubrics.
  • Write clear explanations supporting evaluation scores.
  • Read, validate, and troubleshoot JSON for structural and formatting issues.

🛠️ Requirements

  • C1/C2 spoken fluency in at least one of the target languages listed below.
  • Ability to demonstrate language proficiency through a speaking and listening assessment.
  • Strong JSON skills, including reading, validation, troubleshooting, and identifying structural or formatting errors.
  • Comfort using digital tools and collecting recordings and technical logs.
  • Strong attention to detail and the ability to follow evaluation guidelines consistently.
  • Clear written communication and sound judgment when assessing AI interactions.
  • Experience in AI evaluation, audio annotation, or transcript labeling.
  • A background in technical support, quality assurance, or software development.
  • Familiarity with conversational AI, APIs, and structured data.
Full job description

Help shape the future of AI by evaluating how conversational AI agents communicate and perform across different languages.

  • Location: Remote / Global, subject to project location restrictions

  • Type: Independent contractor / Project-based

  • Commitment: To be confirmed

  • Compensation: To be confirmed

  • Project duration: 2–3 months

About the role

At Anyone AI, we create high-quality training and evaluation data for frontier AI models.

We’re looking for multilingual audio evaluators with strong spoken-language proficiency and technical literacy to assess conversational AI agents.

In this role, you’ll participate in live, real-time roleplay scenarios, capture session recordings and technical logs, label conversation transcripts, and evaluate interactions using structured rubrics. Your feedback will help improve the quality and reliability of AI conversations across languages.

What you’ll do

  • Prepare and confirm the required tools and access before each live session.

  • Conduct live, real-time roleplay scenarios with conversational AI agents.

  • Capture recordings, session IDs, and tool-call or console logs.

  • Label conversation transcripts according to project guidelines.

  • Score interactions against structured evaluation rubrics.

  • Provide clear written explanations supporting your scores.

  • Read, validate, and troubleshoot JSON, identifying structural and formatting issues.

What we’re looking for

  • C1/C2 spoken fluency in at least one of the target languages listed below.

  • Ability to demonstrate language proficiency through a speaking and listening assessment.

  • Strong JSON skills, including reading, validation, troubleshooting, and identifying structural or formatting errors.

  • Comfort using digital tools and collecting recordings and technical logs.

  • Strong attention to detail and the ability to follow evaluation guidelines consistently.

  • Clear written communication and sound judgment when assessing AI interactions.

Languages we’re hiring for

  • Welsh

  • Dutch

  • Greek

  • Italian

  • French

  • German

  • Portuguese

Nice to have

  • Experience in AI evaluation, audio annotation, or transcript labeling.

  • A background in technical support, quality assurance, or software development.

  • Familiarity with conversational AI, APIs, and structured data.

Similar jobs

Isaac Health

Neurology Nurse Practitioner (Active Nebraska license required)

🕒 yesterday
Isaac Health👥 51 - 200 employees🏢 Hospital & Health Care

Provide evidence-based neurology and dementia care through patient evaluations and follow-ups, partnering with therapists and neurologists to coordinate treatment and support patients and families.

📍 🇺🇸 United States - Remote⚖️ Part-Time🎹 Mid-level🎲 Other📢 🇬🇧 English Required📢 🇪🇸 Spanish Required
NeurologyNurse Practitioner CareDementia CareCognitive-behavioral Disorders
Covista

PMHNP (Adjunct)

🕒 yesterday
Covista👥 1001 - 5000 employees🏢 Education Management

Teach and support nursing students in an online learning environment by facilitating course discussions, assessing student work, providing timely feedback, and mentoring learners in psychiatric mental health nursing.

Psychiatric Mental Health NursingPMHNPOnline TeachingCanvas
DENTEGO

Odontólogo(a) en Francia H/M

🕒 yesterday
DENTEGO👥 201 - 500 employees🏢 Medical Practice

El/la odontólogo/a realizará tratamientos dentales polivalentes y podrá colocar implantes o tratar casos con alineadores si cuenta con la formación adecuada, integrándose en un equipo clínico con apoyo dedicado.

OdontologíaProfilaxisEndodonciaCirugía Dental Menor
Greenlife Healthcare Staffing

Telehealth Physician Assistant (Primary Care)

🕒 2 days ago
Greenlife Healthcare Staffing🏢 Healthcare🤝 B2B

Provide virtual primary care by assessing patients, diagnosing and managing acute and chronic conditions, prescribing treatments, and coordinating follow-up while maintaining compliant clinical documentation.

📍 🇺🇸 North Carolina - Remote💰 $62 / hour💼 Full-Time🎹 Mid-level🎲 Other📢 🇬🇧 English Required📢 🇪🇸 Spanish Required
TelehealthPrimary CarePatient AssessmentDiagnosis And Treatment Planning

Trusted by Remote Workers