Remote Jobs RockRemote Jobs Rock

Automation Test Engineer

📅 Sep 11
Automation TestingAI/ML TestingSeleniumCypress
Apply Now

📜 Description

  • Support the development of an AI/ML platform for user burden relief across CMS.
  • Design and maintain robust, scalable automated test frameworks for web apps and APIs.
  • Develop automated pipelines for model validation, bias detection, and drift monitoring.
  • Define and execute test plans including functional, regression, and AI-specific testing.
  • Partner with AI/ML engineers to create test datasets and simulate real-world user behavior.
  • Mentor junior engineers and advocate for ethical AI standards.

🛠️ Requirements

  • 5+ years of experience in software QA/automation engineering, with at least 1 year in AI/ML product testing.
  • Strong proficiency in at least one automation framework (e.g., Selenium, Cypress) and Python.
  • Hands-on experience with testing AI models, including validation metrics and dataset versioning.
  • Familiarity with tools like MLflow or TensorBoard for AI observability.
  • Understanding of model bias and compliance in regulated environments.
  • Experience in Agile DevSecOps environments with modern CI/CD tooling.

✨ Benefits

  • Medical
  • Dental
  • Vision insurance
  • 15 days of paid leave
  • 7 days of sick leave
  • 2 days bereavement leave
  • 11 paid Federal holidays
  • Up to 40 hours for jury duty
  • 401K with 4% employer contribution (and no vesting period)
  • Up to 4 weeks of paid paternity and maternity leave
Full job description
We are Skyward.
That is, a love for people, for improvement, for human advancement through information technology. We are a people-centered business with a desire to serve others. We are diverse and unified; creative and collaborative; a collection of complementary, not competing talents. And though on the surface we remain relaxed, beneath, a torrent of energy links us to our civic tech mission.
We stand by our values, and we won’t compromise on any of them.
Integrity: We’re conscientious, intentional, and empathetic. Our words and actions align. That’s our character. Please don’t ask us to play another part, we’re poor actors.
Compassionate: If we may borrow a quote from Theodore Roosevelt: “No one cares how much you know until they know how much you care.” Because our team is thoughtful and supportive, caring deeply for each other, our clients, and our work, this comes naturally.
Inquisitive: We remain students by failing openly and turning lessons into solutions.
Unconventional: For us, life isn’t what happens outside of work. Work happens inside of life and our culture erases the line often dividing the two.
Authentic: Made possible only because we embody the values listed above. We’re relaxed and fun yet intensely curious and driven. Team members are placed with thought, care, and precision to ensure that Trust, Truth, and Transparency continue to represent our brand.
Because of that, we continue Onward, Upward, and Skyward.

We need an Automation Test Engineer.
Are you the kind of person who can’t look at a machine learning model without wanting to test it six different ways (and then automate all six)? Do your friends sigh when you bring up responsible AI? Maybe they’ve stopped asking what you do for work because the last time you went on a 20-minute tangent about bias in training data and flaky CI pipelines for neural nets.

If that sounds familiar, you might be exactly who we’re looking for.

Come join us if you're motivated to learn from others, to learn from mistakes, to be part of a future-looking and growth-oriented team.

Let's go Skyward together.

We are Skyward.   That is, a love for people, for improvement, for human advancement through information technology. We are a people-centered business with a desire to serve others. We are diverse and unified; creative and collaborative; a collection of complementary, not competing talents. And though on the surface we remain relaxed, beneath, a torrent of energy links us to our civic tech mission.   We stand by our values, and we won’t compromise on any of them.   Integrity: We’re conscientious, intentional, and empathetic. Our words and actions align. That’s our character. Please don’t ask us to play another part, we’re poor actors.    Compassionate: If we may borrow a quote from Theodore Roosevelt: “No one cares how much you know until they know how much you care.” Because our team is thoughtful and supportive, caring deeply for each other, our clients, and our work, this comes naturally.  Inquisitive: We remain students by failing openly and turning lessons into solutions. Unconventional: For us, life isn’t what happens outside of work. Work happens inside of life and our culture erases the line often dividing the two.    Authentic: Made possible only because we embody the values listed above. We’re relaxed and fun yet intensely curious and driven. Team members are placed with thought, care, and precision to ensure that Trust, Truth, and Transparency continue to represent our brand.   Because of that, we continue Onward, Upward, and Skyward. At Skyward, we support flexible working hours and remote opportunities to help maintain a healthy work-life balance for all employees.   Offers of employment with Skyward are contingent upon acceptable results of a background investigation.   Applicants must have the ability to obtain and maintain a Public Trust security clearance due to the nature of our work as a government contractor.

What you’ll do:

  • Be part of a team supporting the development of an AI/ML platform that will leverage generative AI to relieve user burden across the Centers for Medicare and Medicaid Services (CMS).
  • Design and maintain robust, scalable automated test frameworks for web apps, APIs, and AI-driven features.
  • Develop automated pipelines for model validation, bias detection, and drift monitoring in ML-based systems.
  • Define and execute test plans that include functional, regression, performance, and AI-specific testing strategies.
  • Partner with AI/ML engineers to create test datasets and simulate real-world user behavior and edge cases.
  • Implement CI/CD-integrated quality gates to ensure AI releases meet defined reliability and fairness metrics.
  • Mentor junior engineers and help establish best practices in intelligent automation and AI testing.
  • Advocate for ethical AI standards and contribute to cross-functional discussions about responsible AI.
  • What we'd like you to have:

  • 5+ years of experience in software QA/automation engineering, with at least 1 year focused on AI/ML product testing.
  • Strong proficiency in at least one automation framework (e.g., Selenium, Cypress, Playwright) and scripting language (Python preferred).
  • Hands-on experience with testing AI models, including validation metrics, dataset versioning, and model explainability.
  • Familiarity with tools like MLflow, TensorBoard, or similar for AI observability.
  • Understanding of model bias, fairness audits, and compliance considerations in government or regulated environments.
  • Experience working in Agile DevSecOps environments with modern CI/CD tooling (Jenkins, GitHub Actions, GitLab, etc.).
  • Strong collaboration skills with a knack for clear communication across engineering, data science, and product teams.
  • What would blow us away:

  • You’ve previously supported the Centers for Medicare and Medicaid Services (CMS).
  • Even if you don’t meet 100% of the qualifications, we encourage you to apply. At Skyward, we’re focused on hiring individuals with the right skills and passion to grow, not just checking off every box.

    And now the important part. What we offer you:

  • Medical, dental, vision insurance
  • 15 days of paid leave
  • 7 days of sick leave
  • 2 days bereavement leave
  • 11 paid Federal holidays
  • Up to 40 hours for jury duty
  • 401K with 4% employer contribution (and no vesting period)
  • Up to 4 weeks of paid paternity and maternity leave
  • Company provided laptop
  • $5,000 per year for professional development
  • $600 per year for technical supplies and equipment
  • $2,000 referral bonus
  • Life and disability insurance
  • HSA and FSA
  • Legal Shield and ID Shield Voluntary Benefits
  • Opportunity to work in a collaborative, motivated team focused on modernizing government services with cutting-edge technology and innovative solutions. Who says government work can't be exciting!
  • Bitdeer Technologies Group

    Cloud Validation & Release Engineer

    🕒 2 days ago
    Bitdeer Technologies Group👥 201 - 500 employees🏢 Computer Software

    Own regression, compatibility, acceptance, and release validation across GPU SKUs, drivers, and images, building automated quality gates and staged rollouts that help infrastructure releases ship reliably.

    QAValidation EngineeringRelease EngineeringRegression Testing
    Ohme

    QA Engineer (Firmware & Embedded Systems)

    🕒 2 days ago
    Ohme👥 201 - 500 employees🏢 Electrical/electronic Manufacturing

    Own testing for firmware, embedded systems, and hardware-software integrations for a next-generation EV charger, improving product reliability through test planning, framework development, and cross-functional collaboration.

    Firmware TestingEmbedded Systems TestingHardware-software Integration TestingTest Planning
    Seja Globalweb

    ANALISTA DE TESTES PL

    🔥 22 hours ago
    Seja Globalweb👥 1001 - 5000 employees🏢 Information Technology And Services

    Analista de Testes PL define estratégias e executa testes manuais e automatizados, mantendo cenários e scripts, investigando falhas e apoiando o planejamento e acompanhamento dos testes de aceitação.

    Testes Funcionais ManuaisRobot FrameworkCypressTestes Exploratórios
    Institutoeldorado

    Analista de Teste de Software

    🕒 4 days ago
    Institutoeldorado👥 1001 - 5000 employees🏢 Information Technology And Services

    Analista de Teste de Software cria e automatiza testes para aplicações web e sistemas frontend, middle e backend, colaborando com equipes para prevenir defeitos e garantir a qualidade de ponta a ponta.

    Garantia De Qualidade De SoftwareTestes ManuaisAutomação De TestesSelenium

    Own quality checks for a SaaS product by combining exploratory and regression testing with Playwright automation, clear bug reporting, release verification, and validation of billing and customer data migrations.

    Quality AssuranceExploratory TestingRegression TestingPlaywright

    Trusted by Remote Workers