Design and optimize scalable batch and streaming data pipelines, model data, and own reliability, quality, observability, lineage, and cost while translating client and business needs into dependable data products.
Data Engineer Mexico
📜 Description
- Architect and maintain scalable pipelines for AI/ML workflows, feature engineering, and model-training datasets.
- Design data pipelines, data lakes, and data warehousing solutions.
- Partner with data scientists and analysts to deploy machine learning models and analytics on Databricks.
- Optimize Databricks cluster performance, cost, and resource utilization.
- Implement platform security, data governance, and compliance practices.
- Contribute to engineering best practices and share knowledge about Databricks and industry trends.
🛠️ Requirements
- 1-4 years of experience in data engineering or a related field.
- Experience with Databricks, including platform engineering, data engineering, and analytics.
- Proficiency in Python, Scala, or Java, with experience using Databricks APIs, Apache Spark, and Delta Lake.
- Experience with data warehousing, ETL processes, and data governance frameworks.
- Practical experience supporting AI/ML lifecycles, with familiarity with ML frameworks or MLOps practices.
- Ability to work independently, take ownership, collaborate effectively, and communicate with stakeholders.
✨ Benefits
- Learning and growth opportunities.
- Medical insurance for the employee and family.
- Employee rewards program.
- Paid parental leave.
- Flexible schedule.
- Remote working benefits.
Full job description
Role Overview
We are seeking a hands-on Data Engineer to join our team, focusing on building the
robust foundations required for modern analytics and AI. In this role, you will bridge
the gap between raw data and production-ready AI solutions, architecting scalable
pipelines that power machine learning models and business intelligence. You will be
responsible for the engineering discipline that ensures data reliability, performance, and
security within the Databricks ecosystem.
Key Responsibilities
● AI/ML Pipeline Support: Architect, build, and maintain scalable data pipelines
specifically designed to support AI/ML workflows, including feature
engineering and model training datasets.
● Design and implement robust data pipelines, data lakes, and data warehousing
solutions.
● Partner with data scientists and analysts to deploy machine learning models and
analytics solutions on Databricks.
● Enhance Databricks cluster performance, ensuring cost-effectiveness and efficient
resource utilization.
● Implement Databricks security features, enforce data governance, and maintain
compliance with industry standards.
● MLOps Integration: Collaborate with Data Scientists to deploy and
operationalize machine learning models into production environments securely.
● Contribute to the technical direction and best practices of data engineering
initiatives.
● Stay current with Databricks innovations, best practices, and industry trends,
and share knowledge across the team.
Required Skills & Qualifications
● Total Experience: 1-4 years in data engineering or related fields.
● Databricks Expertise: Solid experience with Databricks, focusing on platform
engineering, data engineering, and analytics.
● Technical Skills: Proficiency in Python, Scala, or Java; strong experience with
Databricks APIs, Apache Spark, and Delta Lake.
● Data Engineering Background: Solid experience in data warehousing, ETL
processes, and data governance frameworks.
● AI/ML Exposure: Practical experience supporting AI/ML lifecycles, including
familiarity with ML frameworks (e.g., TensorFlow, PyTorch) or MLOps practices.
● Collaboration & Ownership: Proven ability to work independently, take
ownership of projects, and collaborate effectively with a team.
● Communication: Excellent interpersonal and communication skills for effective
collaboration with stakeholders.
Good to Have
● Cloud Experience: Hands-on experience with AWS, Azure, or GCP.
● Certifications: Databricks Certified Professional or equivalent.
● Machine Learning: Familiarity with ML concepts and popular libraries (e.g.,
TensorFlow, PyTorch, Scikit-learn).
What we are offering
● Best-in-industry learning and growth opportunities.
● Comprehensive Health and wellness benefits, including medical insurance for
self and family.
● Employee rewards program and paid parental leave.
● Flexible schedule and remote working benefits.
Similar jobs
Search more Data Engineer jobsData Engineer junior (con Experiencia) (Chile- Perú)
Diseña, desarrolla y optimiza pipelines y soluciones de procesamiento de datos, participando en migraciones y modernización de plataformas para mejorar la calidad, disponibilidad y escalabilidad de los datos.
Develop and optimize DataLake pipelines and ETL processes, analyze data models and dependencies, and ensure data quality through SQL validation, reconciliation, and production support.
Develop and optimize DataLake pipelines and ETL processes, analyze data models and dependencies, and ensure data quality through SQL validation, reconciliation, and production support.
Data Engineer (DATABRICKS / PYSPARK)
Develop and optimize scalable batch and near real-time data pipelines using Databricks, PySpark, and SQL, supporting reliable Data Lakehouse solutions for large-scale marketing and advertising data.
