Remote Jobs RockRemote Jobs Rock

Machine Learning Engineer, TTS

📅 Apr 29
PytorchTransformer ArchitecturesDiffusion ModelsAudio Language Modeling
Apply Now

📜 Description

  • Architect, train, fine-tune, and align large-scale speech models.
  • Lead research projects and design experiments to improve model performance.
  • Define data requirements and guide acquisition, curation, augmentation, labeling, and synthetic data strategies.
  • Build objective and subjective evaluations, including listening tests, speech metrics, robustness, bias, and red-team checks.
  • Harden training, evaluation, and inference pipelines; optimize latency, memory, and cost for production.
  • Scale distributed training and inference with infrastructure partners and contribute to speech safety and misuse mitigation.

🛠️ Requirements

  • Exceptional research/development experience with large scale audio models (>3B models and >500k hours data).
  • Exceptional understanding and hands-on experience with transformer architectures and/or diffusion models (inc. distillation and streaming) and/or audio language modelling.
  • Strong experience with multi-node and multi-gpu distributed model training.
  • Strong software engineering skills with a proven track record of building complex systems
  • Strong with PyTorch and performance work (profiling, CUDA/Triton/C++ as needed) and writing reliable production quality code.
  • Shipped large scale speech/audio models to production.
  • Background in working with large-scale ML data.
  • Ability to iterate on data,, and triangulate quality using subjective and objective signals.
  • Notable publications and/or open source contributions in speech/audio/ML.
  • Experience with voice-cloning, speech-control, voice-generation.

✨ Benefits

  • Competitive salary and generous company equity
  • Medical
  • Dental
  • Vision insurance – 99.99% of premiums covered by Cantina
  • 42 days of paid time off, including:15 PTO days10 sick days15 company holidays2 floating holidays
  • Generous parental leave & fertility support
  • 401(k) retirement savings plan
  • Lifestyle spending account – $500/month to use however you’d like
  • Complimentary lunch and snacks for in-office employees
  • One Medical membership, and more!
Full job description

About Cantina

Cantina is a new social platform founded by Sean Parker with the most advanced AI character creator. Our bots are lifelike, social creatures that can interact wherever people are online—across voice, video, and text. Create yourself, imagine someone new, or choose from thousands of characters to share infinitely scalable, personalized content and seamless group chat.

If you’re excited about how AI can shape creativity and social interaction, come help us build what’s next.

About the Role:

We’re looking for a Research / ML Engineer to join our Speech Team to build state-of-the-art speech systems end-to-end—from data specs through production inference. You’ll drive the model ↔ data ↔ eval flywheel for TTS and adjacent tasks (voice cloning, controllable TTS, voice conversion and more), partnering closely with research, data, and infra to ship fast, reliable, and cost-aware models. In this role, you will work at the intersection of cutting-edge research and practical engineering, contributing to the development of safe, steerable, and trustworthy AI systems.

What You’ll Do:

  • Model Building: Architect, implement, pre-train, fine-tune, and post-train/alignment (e.g., GRPO/DPO) for large-scale speech models.

  • Project Leadership: Independently lead small research projects while collaborating on larger team initiatives.

  • Experimental Design: Design, run, and analyze scientific experiments to advance our understanding of the models.

  • Tool Development: Develop and improve dev tooling to enhance team productivity.

  • Full-Stack Contribution: Contribute to the entire stack, from low-level optimizations to high-level model design.

  • Data Ownership: Define data requirements and collaborate on acquisition, curation, augmentation, labeling quality, and synthetic data strategies.

  • Rigorous Evaluation: Design automated objective/subjective evaluations—listening tests, SV/WER/ASR-based metrics, robustness & bias checks, and red-team studies.

  • Pipeline Delivery: Harden the training → evaluation → inference pipeline; profile latency, memory, and cost; and meet production SLAs with robust monitoring and rollback.

  • GPU Scaling: Partner with infrastructure to run distributed training/inference on cloud fleets and productionize models with reliability and observability.

  • Safety & Responsibility: Contribute to safety/consent guardrails and to misuse/abuse mitigation for responsible speech technology.

What You’ll Bring:

  • Exceptional research/development experience with large scale audio models (>3B models and >500k hours data).

  • Exceptional understanding and hands-on experience with transformer architectures and/or diffusion models (inc. distillation and streaming) and/or audio language modelling.

  • Strong experience with multi-node and multi-gpu distributed model training.

  • Strong software engineering skills with a proven track record of building complex systems

  • Strong with PyTorch and performance work (profiling, CUDA/Triton/C++ as needed) and writing reliable production quality code.

  • Shipped large scale speech/audio models to production.

  • Background in working with large-scale ML data.

  • Ability to iterate on data,, and triangulate quality using subjective and objective signals.

  • Notable publications and/or open source contributions in speech/audio/ML.

  • Experience with voice-cloning, speech-control, voice-generation.

Preferred Experience:

  • Shipped large scale speech/audio models (TTS/VC/ASR) to production.

  • Work on large-scale ML systems.

  • Experience with audio language modelling, transformer architectures.

  • Experience with voice-cloning, speech-control, voice-generation.

  • Background in processing large-scale ML data.

  • Publications or notable open-source in speech/audio/ML.

Compensation:

When determining compensation, a number of factors will be considered, including skills, experience, job scope, location, and competitive compensation market data.

 

Benefits for U.S.-based roles:

  • Competitive salary and generous company equity

  • Medical, dental, and vision insurance – 99.99% of premiums covered by Cantina

  • 42 days of paid time off, including:

    • 15 PTO days

    • 10 sick days

    • 15 company holidays

    • 2 floating holidays

  • Generous parental leave & fertility support

  • 401(k) retirement savings plan

  • Lifestyle spending account – $500/month to use however you’d like

  • Complimentary lunch and snacks for in-office employees

  • One Medical membership, and more!

Runway-Ml

Member of Technical Staff, ML Platform (Evals)

🕒 3 days ago
Runway-Ml👥 201 - 500 employees🏢 Computer Software🧠 AI-native

Own the ML evaluation platform end to end, building scalable systems to generate, score, and track model results so research teams can make reliable training and release decisions.

PythonPytorchKubernetesML Infrastructure
Coinbase

Senior Machine Learning Engineer

🕒 9 days ago
Coinbase👥 10,000+ employees🏢 Financial Services

Own end-to-end machine learning initiatives powering personalized recommendations, notifications, and LLM-generated market insights, from modeling and evaluation through production serving and experimentation.

Machine Learning EngineeringPythonGoRecommendation Systems
Stack AV

Senior Software Engineer, ML Platform

🕒 4 days ago
Stack AV👥 201 - 500 employees🏢 Computer Software🧠 AI-native

Architect and build ML data platforms for autonomous driving, including multimodal semantic search, data mining, and curation workflows that support training, inference, and safety evaluation.

Machine Learning PlatformsML ApplicationsMultimodal Semantic SearchLarge Language Models

Trusted by Remote Workers