Build and govern Databricks data pipelines for migration reconciliation and reporting, expose validated results through an API, and orchestrate production workflows with data quality controls.
Senior Data Engineer
📜 Description
- Design, develop, and deploy scalable Databricks data pipelines and ETL workflows.
- Transform and harmonize data from APIs, files, databases, and cloud platforms into analytics-ready datasets.
- Build and industrialize cloud-based data solutions for data ingestion, integration, processing, and delivery.
- Define data architectures and implement quality assurance, validation, and monitoring processes.
- Translate client needs into scalable engineering designs and communicate recommendations and project progress.
- Own project deliverables, timelines, priorities, and risks while coordinating cross-functional teams.
🛠️ Requirements
- Bachelor’s or Master’s degree in Data Science, Computer Science, Engineering, or a related quantitative field.
- 4-6 years of experience in data engineering, analytics, or statistical modeling.
- Proficiency with Databricks, including Delta Lake, Spark, Asset Bundles, Unity Catalog, and cluster management.
- Proficiency in Python and working knowledge of SQL, Spark, and Bash scripting.
- Experience building production-grade data pipelines and architectures, including CI/CD and ETL/ELT processes.
- Experience with Azure; familiarity with GCP or AWS is a plus.
- Experience with Power BI or Tableau; exposure to machine learning or advanced analytics is a plus.
- Strong communication, stakeholder management, and project delivery skills, including explaining technical concepts to non-technical audiences.
Full job description
Ekimetrics is a global leader in Marketing & Commercial effectiveness and AI-powered solutions. Since 2006, we’ve been helping businesses optimize marketing and operations by combining AI with business and tech expertise across 4 domains: Marketing & commercial effectiveness, Customer analytics, Operational excellence, ESG & sustainability.
With a full in-house team and offices in Paris, London, New York, Hong Kong, and Shanghai, we deliver tailor-made solutions that turn data into real positive impact, leveraging our unique combination of technology and services.
We excel at delivering AI impact at scale. Our mission is to harness this power to tackle the world’s most urgent issues. We commit to responsibility and frugality in AI, systematic AI impact at scale, and loyalty to our values and DNA.
Your Responsibilities
As a Senior Data Engineer, you will lead and contribute to high-impact projects with top-tier international clients across industries, designing, implementing, and optimizing ETL solutions that support Marketing Mix Modeling and Optimization and broader Marketing Effectiveness initiatives. You’ll work closely with data engineers, data scientists, and tech leads across 1-2 concurrent projects, build robust, scalable data architectures that enable marketing analytics and support data driven decision making. In addition to technical ownership, you will play a key role in project and stakeholder management, partnering with cross-functional teams and client stakeholders to ensure successful project delivery and business impact. You will also benefit from our technology partnerships, training programs, and mentorship from Ekimetrics partners, further developing your expertise in data engineering, project management, and stakeholder engagement. Your responsibilities will include:
Data Engineering
- Design, develop, and deploy scalable data pipelines and ETL workflows in Databricks.
- Transform and harmonize data from diverse sources and formats (i.e. APIs, CSV files, JSON, databases, cloud platforms, file transfers, etc.)
- Build and industrialize cloud-based data solutions and frameworks that support efficient data ingestion, integration, processing, and delivery, enabling reliable and analytics-ready datasets for accounts.
- Define and implement robust data architectures, QA, validation, and monitoring processes to ensure accuracy, consistency, reliability, and scalability.
- Partner with internal and external stakeholders to understand client data ecosystems, business objectives, account needs, and long-term goals, ensuring alignment between solutions and business priorities.
- Translate client requirements into scalable data engineering designs, collaborating cross-functional teams to develop effective ETL frameworks.
- Serve as a trusted point of contact through project delivery, communicating technical recommendations and project progress while promoting best practices in data management.
- Take ownership of project deliverables from planning through execution, managing priorities, timelines, and risks to ensure high-quality outcomes delivered on time aligned with client objectives.
- Proactively identify and mitigate potential project, technical, and data-related risks, communicating issues, recommendations, and status updates clearly.
- Collaborate cross-functionally to coordinate activities, align expectations, and drive project execution.
Stakeholder Management
Project Management & Delivery Excellence
Your Profile
- Bachelor’s or Master’s degree in Data Science, Computer Science, Engineering, or a related quantitative field, with 4-6 years of experience in data engineering, analytics, or statistical modeling.
- Proficient with Databricks (Delta Lake, Spark, Asset Bundles, Unity Catalog, Cluster Management)
- Proficiency in Python and working knowledge of SQL, Spark, and Bash scripting.
- Experience with templating languages is a plus.
- Skilled in building and deploying production-grade data pipelines and architectures complete with CI/CD plus experience with ELT/ETL processes.
- Experience with Azure required, familiarity with GCP or AWS a plus.
- Experience with PowerBI or Tableau for dashboarding and data visualizations.
- Exposure to ML or advanced analytics techniques is a plus.
- Strong communication, stakeholder management, and project delivery skills, with the ability to explain technical concepts to non-technical audiences.
Similar jobs
Search more Data Engineer jobsDrive Databricks adoption and technical standards while designing and optimizing scalable data pipelines, strengthening governance and observability, and guiding engineers across data engineering initiatives.
Own the full lifecycle of data products, from analysis and design through deployment, while building PySpark programs and aligning technical solutions with evolving business needs.
Design and optimize scalable data pipelines and cloud-native architectures that support Data & AI initiatives, collaborating across teams to deliver secure, high-performing data solutions.
As a Senior Data Engineer, you will build, enhance, and maintain our data platform on Databricks, transforming large datasets into clean, usable data products.
