Description
Builds and runs large scale batch data pipelines, keeping jobs fast and affordable as data volumes grow. Works with Apache Spark to build and manage large scale data processing jobs, including performance tuning to maintain efficient and reliable pipeline execution. Supports data modelling for analytics and organises data in a lakehouse so it is usable downstream. Handles automated scheduling, monitoring, and data quality checks, while working with platform and product teams on end to end data flows.
Requirements
Requirements
Designs and maintains AWS data pipelines, models, and integrations while translating stakeholder needs into technical requirements and ensuring reliable, secure data across systems.
Design, build, and optimize scalable data platforms as a Senior Data Engineer, focusing on analytics, ecommerce, logistics, and marketing initiatives.
Own the end-to-end data platform, designing multi-source ingestion, event-driven processing, and orchestrated workflows while ensuring reliable storage, transformation, validation, and production operations.
Own the end-to-end data platform, designing multi-source ingestion and event-driven workflows, building reliable transformation pipelines, and operating production data infrastructure.
Own the end-to-end data platform, designing multi-source ingestion, event-driven processing, and orchestrated workflows that deliver reliable, trusted data for downstream products.
