Design and implement specialized data collection infrastructure to support Centerfield's proprietary Data Architecture while troubleshooting and resolving technical issues.
Data Engineer Semi Senior
🕒 18 days ago
Google Cloud PlatformBigqueryDataformPub/sub
📜 Description
- Design, develop, and maintain reliable and scalable data pipelines using Google Cloud Platform tools.
- Integrate various data sources through REST API consumption and exposure, ensuring proper data ingestion and availability.
- Manage and transform cloud data using BigQuery, Dataform, and Cloud Storage, ensuring consistency and quality.
- Implement real-time messaging and event processing flows via Pub/Sub, coordinating service orchestration.
- Document processes, data architectures, and technical decisions for maintainability and knowledge transfer.
- Collaborate with Analytics, Engineering, and Product teams to understand business requirements and translate them into efficient data solutions.
🛠️ Requirements
- Minimum 3 years of experience in Data Engineering roles, actively participating in pipeline implementation projects.
- Proficiency in Google Cloud Platform, including BigQuery, Dataform, Pub/Sub, and Cloud Storage.
- Strong Python skills for developing transformation scripts, process automation, and API consumption.
- Advanced SQL capabilities for modeling, complex queries, query optimization, and building analytical models in BigQuery.
- Experience integrating with REST APIs: authentication, pagination, error handling, and data normalization.
- Knowledge of data engineering best practices: code versioning, pipeline testing, dependency management, and technical documentation.
Full job description
Buscamos un Data Engineer con sólida experiencia en el ecosistema de Google Cloud Platform, capaz de diseñar, desarrollar y mantener pipelines de datos robustos y escalables. El rol implica trabajar en proyectos de implementación donde la calidad, trazabilidad y eficiencia del dato son prioridad. Si te apasiona transformar datos en activos estratégicos y disfrutas resolver desafíos técnicos complejos en entornos cloud, esta oportunidad es para ti.
Responsabilidades principales
Diseñar, desarrollar y mantener pipelines de datos confiables y escalables utilizando las herramientas del ecosistema Google Cloud Platform.
Integrar diversas fuentes de datos mediante el consumo y exposición de APIs REST, garantizando la correcta ingestión y disponibilidad de la información.
Gestionar y transformar datos en la nube utilizando BigQuery, Dataform y Cloud Storage, asegurando consistencia, calidad y rendimiento en cada capa del modelo de datos.
Implementar flujos de mensajería y procesamiento de eventos en tiempo real a través de Pub/Sub, coordinando la orquestación entre servicios.
Documentar los procesos, arquitecturas de datos y decisiones técnicas para garantizar la mantenibilidad y transferencia de conocimiento en el equipo.
Colaborar con equipos de Analytics, Ingeniería y Producto para comprender los requerimientos de negocio y traducirlos en soluciones de datos eficientes.
Similar jobs
Search more Data Engineer jobs
As a Data Engineer, you will build reliable data systems and pipelines that empower teams at Stake to make informed decisions and enhance operational efficiency.
As a Data Engineer, you will build and optimize data pipelines, support analytics tooling, and extend the machine learning platform while mentoring junior engineers.
Data Engineer
Sigmasoftware2👥 1001 - 5000 employees🏢 Computer Software
As a Data Engineer, you will design and maintain scalable ETL/ELT pipelines and optimize data processing solutions for a high-traffic E-commerce platform.
Data Developer - Robotics IS Data Platform
Fusionconsulting👥 201 - 500 employees🏢 Information Technology And Services
The Data Developer is responsible for designing and maintaining data integration and analytics solutions, ensuring reliable access to enterprise data and supporting business intelligence initiatives.