Remote Jobs RockRemote Jobs Rock

Software Test Engineer

🕒 8 days ago
📍 🇺🇸 United States - Remote💰 $150k - $220k / year🎹 Mid-level💻 Software Engineer💼 Full-Time
Automated TestingTest FrameworksRegression TestingAPI Testing
Apply Now

📜 Description

  • Define and execute well-designed test plans across Deepgram's products, APIs, SDKs, model-powered features, and data platforms.
  • Design, build, and maintain automated test suites and frameworks for functional, integration, end-to-end, regression, API, browser, and service-level testing.
  • Translate product requirements and customer acceptance criteria into clear test strategies and repeatable test cases.
  • Build and maintain customer-focused and adversarial test datasets and environments that exercise real-world inputs and edge cases.
  • Validate model-powered behavior using appropriate metrics and regression coverage, partnering with Research and model-evaluation specialists.
  • Integrate automated tests and quality checks into CI/CD to ensure continuous regression detection.

🛠️ Requirements

  • BS, MS, or PhD in Computer Science, AI, Applied Math, or a related field, or equivalent experience.
  • 5+ years of professional software or QA engineering experience, with a track record of shipping test infrastructure or evaluation systems (senior candidates with significantly deeper experience welcome).
  • Solid backend/scripting experience in a language such as Python, Rust, Go, or similar.
  • Experience designing and building automated test pipelines, evaluation frameworks, or data-processing systems.
  • Strong analytical skills and comfort reasoning about metrics, thresholds, and statistical variation in results — able to distinguish real regressions from noise.
  • Ability to take charge of ambiguous technical challenges and communicate effectively across research, engineering, and product teams.
  • Hands-on experience testing or evaluating modern AI systems such as LLMs, RAG pipelines, agents, or multimodal models, including analyzing model behavior and failure modes.
  • Experience with voice, audio, speech recognition, or real-time systems, and familiarity with metrics such as WER, MOS, latency, and time-to-first-byte.
  • Experience building or improving test, evaluation, benchmarking, or ML infrastructure used by multiple teams or external users.
  • A strong appreciation for test and evaluation quality, including correctness, reproducibility, determinism, and consistency across environments.
Apply Now

Similar jobs

Search more Software Engineer jobs
Pinterest

SDET II, tvScientific

Pinterest👥 5001 - 10,000 employees🏢 Computer Software
WebsiteLinkedInAll Job Openings
🔥 15 hours ago

tvScientific is seeking an SDET Engineer to ensure the quality of its web platform through manual and automated testing, focusing on end-to-end user flows across UI, APIs, and backend services.

📍 🇺🇸 San Francisco - Remote💰 $109.2k - $224.9k / year💼 Full-Time🎹 Mid-level💻 Software Engineer
Web TestingAPI TestingManual TestingAutomation Testing
View jobApply
Reddit

Software Engineer - Data Movement Platform

Reddit👥 1001 - 5000 employees🏢 Online Community/social Media
WebsiteLinkedInAll Job Openings
🔥 12 hours ago

As a Software Engineer on the Data Movement team, you will enhance scalable data infrastructure and streamline data processing for machine learning and analytics at Reddit.

📍 🇺🇸 United States - Remote💰 $164.2k - $229.9k / year💼 Full-Time🎹 Mid-level💻 Software Engineer
Software EngineeringData InfrastructureMachine LearningAnalytics
View jobApply
View more Software Engineer jobs

Trusted by Remote Workers