Build production-ready data pipelines that ingest, process, and transform client data into scalable datasets powering AI agents, while translating business needs into reliable solutions.
Senior Data Engineer
📜 Description
- Architectural Leadership: Lead the design and development of scalable, high-performance data pipelines and infrastructure that power Samba TV's analytics.
- Complex Problem Solving: Resolve complex technical issues in creative and effective ways, understanding the interrelationships of different disciplines.
- Production Dataset Management: Lead the design, build, and maintenance of high-scale production datasets, ensuring reliable customer-facing reports.
- Schema Evolution & Lifecycle: Manage the end-to-end lifecycle of data features, including complex backfills and reprocessing.
- Performance & Cost Engineering: Architect efficient solutions for data ingestion, transformation, and storage while driving down costs.
- Incident Response & Debugging: Serve as a technical lead for production incidents, investigating root causes and implementing fixes.
🛠️ Requirements
- Experience: Typically 8+ years of related experience with a Bachelor’s degree (or 6 years with a Master’s; 3 years with a PhD).
- Technical Mastery: Advanced knowledge of Python and deep understanding of distributed data processing frameworks like Apache Spark or PySpark.
- Orchestration Expertise: Must-have expertise in Apache Airflow and Databricks for orchestration and scalable data processing.
- Infrastructure: Extensive experience with cloud-based data infrastructure (AWS, GCP) and modern data lake architectures.
- Data Modeling: Strong knowledge of data modeling, database design, and query optimization for both relational and non-relational databases.
- Software Excellence: Proven track record of driving best practices for code quality, testing, and software design in a data engineering context.
- Communication: Ability to adapt your communication style and use persuasion when delivering messages that relate to the wider firm business.
Full job description
What You'll Do
Who You Are
Similar jobs
Search more Data Engineer jobsOwn and evolve the data warehouse platform end-to-end, building reliable CDC pipelines, lakehouse and query layers, transformations, and analytics tooling while setting architecture and platform standards.
Build and scale the data and analytics platform behind clinical AI products, developing secure pipelines, performance monitoring, and self-serve tools that turn complex clinical data into actionable insights.
Design and optimize scalable Databricks data pipelines and infrastructure, applying data governance practices to improve data quality, lineage, security, and reliability across enterprise data platforms.
Senior Data Engineer (Databricks) - Contractor - US
Build and optimize Databricks data platforms and pipelines across batch and streaming workloads, contribute technical leadership and solution accelerators, and help deliver scalable client data solutions.
