Staff Data Engineer

This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a Staff Data Engineer based in the United States.

As a Staff Data Engineer, you will shape enterprise-wide data architecture and build scalable, reliable data platforms that support critical business and analytical needs. You will provide technical leadership across data engineering initiatives, influencing architectural decisions, engineering standards, and long-term platform strategy. Working with cross-functional teams, you will design and optimize complex data pipelines for large-scale batch and streaming workloads. You will also strengthen data quality, governance, monitoring, and deployment automation to ensure trustworthy and accessible data. Combining advanced technical expertise with strategic thinking, you will help improve how teams process, analyze, and use information. This fully remote opportunity is suited to an experienced data engineering professional who thrives on solving complex technical challenges and driving organization-wide impact.

You will lead data architecture and platform design while developing robust data pipelines, improving engineering practices, and ensuring the reliability and performance of enterprise data systems.

• Lead enterprise-wide data architecture strategy and define long-term data platform design principles.
• Evaluate and approve major technical designs, establish engineering best practices, and advise leadership on data technology decisions and roadmap priorities.
• Champion data governance, data democratization, and scalable standards that improve data accessibility, consistency, and quality.
• Collaborate with Solutions Delivery, Business Intelligence, Predictive Analytics, Enterprise Services, and other cross-functional teams to resolve technical data challenges.
• Design, develop, and optimize ETL pipelines using Apache Airflow, dbt, Python, and Snowflake to ingest, transform, and load data into analytical warehouses.
• Build and manage large-scale data ingestion and transformation workflows for structured and unstructured datasets.
• Develop streaming pipelines using Confluent Kafka to support event-driven processing and real-time analytics.
• Use Spark and other big data technologies to support batch processing, streaming analytics, and large-volume data workloads.
• Build and maintain CI/CD pipelines and automate code and process deployments to improve engineering efficiency and release reliability.
• Establish enterprise-wide data quality frameworks, testing procedures, and validation processes to ensure data accuracy and consistency.
• Monitor pipeline health, freshness, availability, performance, and service-level agreement (SLA) compliance through dashboards and automated monitoring.
• Test, validate, and refine data models and analytical processes to improve accuracy and reliability.
• Analyze large datasets using statistical and programming tools to generate insights and support reporting requirements.
• Optimize cloud infrastructure and containerized workloads to improve scalability, operational reliability, and resource utilization.

Requirements:

The ideal candidate combines advanced academic training in a relevant technical discipline with extensive hands-on experience in data engineering, cloud infrastructure, streaming systems, and automated software delivery.

• A master's degree or foreign equivalent in Data Science, Applied Computer Science, Computer Science, or a related field.
• At least 4 years of experience in each of the following areas:
  • Developing and optimizing scalable data pipelines using Apache Airflow and Python.
  • Building streaming data pipelines using Kafka.
  • Tuning big data systems using Kafka and Hive.
  • Using Apache Spark to create jobs for batch processing and streaming analytics.
  • Writing complex SQL queries to retrieve data from warehouses for analysis and reporting.
  • Using AWS for data processing and large-volume workloads.
  • Automating code and process deployments using Python.
  • Working with cloud-based infrastructure.
  • Building Datadog dashboards to monitor data pipelines and system performance.
  • Using Docker and Kubernetes to improve and validate containerization and automation reliability.
  • Implementing Continuous Integration and Continuous Deployment (CI/CD) pipelines using Jenkins.
  • Using GitHub as a code repository and working with both Windows and Linux operating systems.
  • Strong understanding of enterprise data architecture, data modeling, data quality, and pipeline reliability.
  • Ability to evaluate technical designs, establish engineering standards, and make decisions based on scalability, performance, and maintainability.
  • Experience collaborating with cross-functional teams and communicating technical concepts to technical and non-technical stakeholders.
  • Strong analytical and problem-solving skills, with a focus on building reliable, maintainable, and efficient data systems.
  • Ability to work independently and provide technical guidance on complex, organization-wide engineering initiatives.

  Benefits:

  • Annual salary ranging from $175,000 to $200,000.
  • Fully remote work arrangement, with telecommuting available anywhere in the United States.
  • Paid parental leave.
  • 401(k) retirement plan with employer matching contributions.
  • Flexible vacation time.
  • Medical, dental, and vision coverage.
  • Commuter benefits.