Remote Jobs RockRemote Jobs Rock

Lead Quality Engineer

📅 Sep 23
📍 🇮🇳 Bengaluru - Hybrid🎺 Lead🧳 QA Engineer💼 Full-Time
Quality EngineeringGenai Agent TestingPythonPytest
Apply Now

📜 Description

  • Design and execute functional, integration, regression, security, and performance tests for GenAI agents and multi-agent workflows.
  • Validate application and infrastructure code interrogation findings for accuracy, evidence, severity, and false positives or negatives.
  • Test agent instructions, tool usage, routing, handoffs, error handling, retries, timeouts, and fallback behavior.
  • Validate GitLab Duo and CI/CD integrations, including pipeline triggers, security gates, reports, artifacts, permissions, and auditability.
  • Build Python-based automated test suites and reusable datasets to assess agent reliability, consistency, security, and quality.
  • Test finding triage, deduplication, categorization, prioritization, and correlation across application and infrastructure agents.

🛠️ Requirements

  • Experience with CI/CD pipelines and GitLab integrations.
  • Python test automation and PyTest.
  • Knowledge of GenAI agents, agentic workflows, and evaluation datasets.
  • Application and infrastructure security testing, including OWASP Top 10 risks.
  • Familiarity with Terraform, Kubernetes, Helm, YAML, and CI/CD configuration.
  • Knowledge of REST APIs or GraphQL, Docker, and databases such as PostgreSQL, MongoDB, or Redis.
  • Frontend and backend familiarity with React and Django, Flask, or FastAPI.
Full job description
Consultant

Primary Skills

  • CI/CD Pipeline, React (React Context / Redux), Django / Flask / FastAPI, Docker, CSS/SCSS/LESS, PyTest, PostgreSQL / MongoDB / Redis, Kubernetes, REST API / GraphQL

    • Job requirements

  • Role: QA Engineer – GenAI Agents and GitLab Duo • Design and execute functional, integration, regression, security, and performance tests for GenAI agents and multi-agent workflows. • Validate application and infrastructure code interrogation results for accuracy, evidence, severity, false positives, and false negatives. • Test GitLab Duo, AI Catalog agents, shared CI/CD templates, pipeline triggers, reports, and security gates. • Develop automated Python-based test suites and evaluation datasets to assess agent reliability, consistency, security, and overall quality. Key Responsibilities 1. AI Agent and Workflow Testing • Create and execute test plans for GenAI agents and agentic workflows. • Validate single-agent, multi-agent, and orchestrated workflows. • Verify agent instructions, tool usage, task routing, handoffs, and termination conditions. • Test agent behaviour for valid, invalid, incomplete, and ambiguous inputs. • Validate error handling, retry mechanisms, timeouts, and fallback scenarios. 2. Application Code Interrogation Testing Validate the application code interrogation agent against: • Security vulnerabilities and OWASP Top 10 risks. • Insecure coding practices. • Improper authentication and authorization. • Sensitive-data exposure and secret leakage. • Missing input validation and error handling. • Resiliency issues such as missing timeouts, retries, bulkheads, and circuit breakers. • Incorrect logging, monitoring, and exception-handling practices. Verify that each finding contains accurate: • File name and line number. • Issue description and technical evidence. • Severity and confidence. • Business impact. • Recommended remediation. • Classification as confirmed or potential finding. 3. Infrastructure Code Interrogation Testing Test the infrastructure interrogation agent against Terraform, Kubernetes, Helm, YAML, and CI/CD configuration files. Validate detection of: • Single points of failure. • Inadequate high availability and scaling. • Insecure storage and missing encryption. • Excessive network access. • Unrestricted communication between security zones. • Missing identity and access controls. • Inadequate logging, monitoring, and alerting. • Insecure secrets management. • Misconfigured cloud and container resources. 4. SDLC Agent Validation Support testing of AI agents used across the software development lifecycle, including: • Requirement and story decomposition. • UX/UI specification generation. • OpenAPI contract generation and governance. • Architecture validation. • Backend and frontend implementation assistance. • Dependency and open-source package curation. • Unit, integration, contract, and end-to-end test generation. • Security review. • Infrastructure planning and implementation. 5. Triage and Finding Validation Validate the interrogation-triage agent for: • Finding normalization. • Duplicate detection and deduplication. • Correct categorization. • Severity and priority assignment. • Ranking based on exploitability, exposure, and blast radius. • Correlation of findings from application and infrastructure agents. • Accurate consolidation of related findings. • Avoidance of false positives and unsupported conclusions. 6. GitLab Dedicated CI/CD Integration Testing • Validate agent execution through GitLab Dedicated pipelines. • Test integration with shared CI/CD templates. • Validate pipeline triggers for pushes, merge requests, and scheduled executions. • Verify pipeline status, security gates, and failure behavior. • Validate generated reports, artifacts, logs, and auditability. • Test repository permissions and agent access controls. • Verify behavior across different programming languages, repository structures, and infrastructure projects. 7. Test Automation • Develop automated test cases using Python and suitable testing frameworks. • Build reusable test datasets containing secure, vulnerable, and intentionally incorrect code. • Automate regression testing for prompts, tools, agents, and
    • Apply Now

    Similar jobs

    Search more QA Engineer jobs
    Alice-Bob

    Staff Quality Engineer

    🕒 5 days ago
    Alice-Bob👥 201 - 500 employees🏢 Computer Hardware
    WebsiteLinkedInAll Job Openings

    Establish quality engineering for the firmware production lane, owning system-level test automation and customer acceptance evidence while setting cross-team standards for reliable releases.

    📍 🇫🇷 Paris - Hybrid💼 Full-Time🎺 Lead🧳 QA Engineer📢 🇬🇧 English Required📢 🇫🇷 French Required
    PythonTest AutomationQuality EngineeringAPI Testing
    View jobApply
    Zscaler

    Principal Software Development Engineer

    🕒 12 days ago
    Zscaler👥 10,000+ employees🏢 Computer And Network Security🤝 B2B
    WebsiteLinkedInAll Job Openings

    Validates end-to-end Zero Trust Branch solutions for customer deployments by building realistic test environments, automating functional and scale testing, and resolving complex issues to improve release quality and readiness.

    📍 🇺🇸 Santa Clara - Hybrid💰 $185.5k - $265k / year💼 Full-Time🎺 Lead🧳 QA Engineer
    Zero Trust BranchNetworkingTCP/IPDNS
    View jobApply
    Syndigo

    Lead Quality Engineer

    📅 Sep 24
    Syndigo👥 501 - 1000 employees🏢 Computer Software
    WebsiteLinkedInAll Job Openings

    Leads quality across the software development lifecycle for enterprise SaaS products, owning testing strategy and processes while building AI-enabled automation frameworks for API, functional, and UI testing.

    📍 🇮🇳 Bengaluru - Hybrid💼 Full-Time🎺 Lead🧳 QA Engineer
    Quality EngineeringSoftware TestingTest Automation FrameworksAI And Generative AI
    View jobApply
    Familiarmachines

    Principal Reliability and Systems Test Engineer

    🕒 2 days ago
    Familiarmachines🏢 Robotics And Artificial Intelligence
    WebsiteAll Job Openings

    Leads reliability and systems testing for electromechanical robots, defining test strategy and building lab capabilities, automation, and failure-analysis processes to improve product dependability throughout its lifetime.

    📍 🇺🇸 Boston - Hybrid💼 Full-Time🎺 Lead🧳 QA Engineer
    Reliability TestingSystems TestingProduct ValidationElectromechanical Systems
    View jobApply
    View more QA Engineer jobs

    Trusted by Remote Workers