Remote Jobs RockRemote Jobs Rock

AI Safety Experts - English & Finnish

📅 Sep 16
📍 🇺🇸 United States - Remote💰 $48 - $62 / hour🎹 Mid-level🧠 AI Research Scientist📢 🇬🇧 English Required📢 🇫🇮 Finnish Required💼 Full-Time
AI Red TeamingConversational AI TestingJailbreakingPrompt Injection
Apply Now

📜 Description

  • Red-team conversational AI models and agents using jailbreaks, prompt injections, misuse cases, bias exploitation, and multi-turn manipulation.
  • Annotate model failures, classify vulnerabilities, and flag systemic risks.
  • Apply taxonomies, benchmarks, and playbooks to keep testing consistent.
  • Document findings in actionable reports, datasets, and reproducible attack cases.
  • Review text-based AI outputs involving sensitive topics such as bias, misinformation, and harmful behaviors.

🛠️ Requirements

  • You bring prior red teaming experience (AI adversarial work, cybersecurity, socio-technical probing)
  • You’re curious and adversarial: you instinctively push systems to breaking points
  • You’re structured: you use frameworks or benchmarks, not just random hacks
  • You’re communicative: you explain risks clearly to technical and non-technical stakeholders
  • You’re adaptable: thrive on moving across projects and customers
Full job description

This role is for one of our clients

Compensation: $48 - $62 per hour

Fluent Language Skills Required: English & Finnish. Native fluency in English and Finnish is required for this position.

Why This Role Exists

We are assembling a red team for this project - human data experts who probe AI models with adversarial inputs, surface vulnerabilities, and generate the red team data that makes AI safer for our customers.

This project involves reviewing AI outputs that touch on sensitive topics such as bias, misinformation, or harmful behaviors. All work is text-based, and participation in higher-sensitivity projects is optional and supported by clear guidelines and wellness resources. Before being exposed to any content, the topics will be clearly communicated.

Requirements

What You’ll Do

  • Red team conversational AI models and agents: jailbreaks, prompt injections, misuse cases, bias exploitation, multi-turn manipulation
  • Generate high-quality human data: annotate failures, classify vulnerabilities, and flag systemic risks
  • Apply structure: follow taxonomies, benchmarks, and playbooks to keep testing consistent
  • Document reproducibly: produce reports, datasets, and attack cases customers can act on

Who You Are

  • You bring prior red teaming experience (AI adversarial work, cybersecurity, socio-technical probing)
  • You’re curious and adversarial: you instinctively push systems to breaking points
  • You’re structured: you use frameworks or benchmarks, not just random hacks
  • You’re communicative: you explain risks clearly to technical and non-technical stakeholders
  • You’re adaptable: thrive on moving across projects and customers

Nice-to-Have Specialties

  • Adversarial ML: jailbreak datasets, prompt injection, RLHF/DPO attacks, model extraction
  • Cybersecurity: penetration testing, exploit development, reverse engineering
  • Socio-technical risk: harassment/disinfo probing, abuse analysis, conversational AI testing
  • Creative probing: psychology, acting, writing for unconventional adversarial thinking

What Success Looks Like

  • You uncover vulnerabilities automated tests miss
  • You deliver reproducible artifacts that strengthen customer AI systems
  • Evaluation coverage expands: more scenarios tested, fewer surprises in production
SD Solutions

SafetyTech Client #1 | Adversarial Task Writer for AI Security RL Gyms

🕒 3 days ago
SD Solutions👥 201 - 500 employees🏢 Staffing And Recruiting

Design YAML-based adversarial prompt injection scenarios, test them against frontier models, and validate successful tasks that assess whether AI agents can be manipulated into violating safety policies.

Prompt InjectionAdversarial TestingYAMLDocker

Evaluate AI-generated finance, audit, insurance, compliance, and legal content by reviewing its accuracy and reasoning, identifying gaps, and providing structured feedback to improve model performance.

AI EvaluationFinancial AnalysisInsurance RiskAuditing

Evaluate AI-generated finance, audit, insurance, compliance, and legal content by reviewing its accuracy and reasoning, identifying gaps, and providing structured feedback to improve model performance.

AI EvaluationCritical ThinkingAnalytical ReasoningWritten Communication
Tipalti

Applied AI Scientist

🕒 9 days ago
Tipalti👥 1001 - 5000 employees🏢 Financial Services

Design, build, and deploy production-grade AI agents that automate complex business workflows, owning agent architecture, knowledge systems, evaluation, and cross-functional delivery across enterprise functions.

AI/ML EngineeringAI AgentsAgentic WorkflowsPython
Gramian Consulting Group

AI Conversation Evaluation & Context Research Specialist

🕒 3 days ago
Gramian Consulting Group👥 1 - 10 employees🏢 Information Technology And Services

Evaluate AI conversations for whether they gather decision-relevant context, identify missing information and useful follow-up questions, and provide structured feedback to improve conversational quality.

📍 🇧🇷 Brazil +5 - Remote?💼 Full-Time🎹 Mid-level🪡 Research Analyst
AI Conversation EvaluationQualitative ResearchContext AssessmentInterviewing

Trusted by Remote Workers