🔥 21 hours ago
Own the architecture, ingestion, transformation, and production operations of a multi-source data platform, delivering reliable, validated data for downstream products and company intelligence.
Senior Data Engineer
🔥 21 hours ago
PythonSQLPostgreSQLDbt
📜 Description
- Design and build a multi-source platform for concurrent ingestion, event-driven processing, and orchestrated batch workflows.
- Ingest provider APIs, webhooks, files, and scraped data with rate limiting and robust failure handling.
- Orchestrate scheduled workflows and historical backfills using tools such as Airflow, Dagster, or Prefect.
- Build event-driven systems with messaging and task queues, retries, dead-letter queues, and controlled replay.
- Preserve source payloads in cloud storage and use dbt to transform data into raw, standardized, and curated layers.
- Implement idempotent upserts, data contracts, validation, pipeline tests, monitoring, and optimized PostgreSQL loading.
🛠️ Requirements
- At least 5 years building and operating production data platforms that integrate multiple sources, with ownership from architecture through ongoing operations.
- Production experience with dbt and Google Cloud Platform, plus workflow orchestration using Airflow, Dagster, Prefect, Temporal, or a comparable tool; Airflow is preferred.
- Strong Python skills, including asynchronous HTTP and concurrent processing with asyncio, aiohttp, or HTTPX.
- Strong SQL and PostgreSQL skills, including data modeling, query optimization, and reliable incremental loading.
- Experience building event-driven systems with messaging or task queues, asynchronous processing, and failure recovery.
- Experience with raw and curated storage, retention policies, data validation, pipeline testing, and monitoring.
- Startup or small-team experience and broad ownership are valuable; web scraping, entity resolution, data enrichment, LLM-based extraction, BigQuery, and Dataform experience are useful.
✨ Benefits
- Base salary: $80,000 to $120,000 USD annually. Visa sponsorship is not available.
Full job description
About the Role
Join a data infrastructure and analytics team building a platform that turns information from multiple sources into reliable, up-to-date company intelligence. You will own the platform end to end, from architecture and ingestion through production operations, supporting products with trusted data.
What You'll Do
Design and build a multi-source data platform for concurrent ingestion, event-driven processing, and orchestrated batch workflows.
Ingest data from provider APIs, webhooks, files, and web scraping, with rate limiting and robust failure handling.
Orchestrate scheduled workflows and historical backfills with tools such as Airflow, Dagster, or Prefect.
Build event-driven systems with messaging and task queues, including retries, dead-letter queues, and controlled replay.
Preserve original payloads in cloud storage and transform them with dbt into raw, standardized, and curated layers.
Implement idempotent upserts, data contracts, automated validation, pipeline tests, and monitoring for freshness and failures.
Design and optimize PostgreSQL schemas and loading patterns, and integrate enrichment providers and LLM-based structured extraction.
What We're Looking For
At least 5 years building and operating production data platforms that integrate multiple sources, with ownership from architecture through deployment and ongoing operations.
Production experience with dbt and Google Cloud Platform, plus workflow orchestration using Airflow, Dagster, Prefect, Temporal, or a comparable tool. Airflow experience is preferred.
Strong Python skills, including asynchronous HTTP and concurrent processing with tools such as asyncio, aiohttp, or HTTPX.
Strong SQL and PostgreSQL skills, including data modeling, query optimization, and reliable incremental loading.
Experience building event-driven systems with messaging or task queues, asynchronous processing, and failure recovery.
Experience with raw and curated storage, retention policies, data validation, pipeline testing, and monitoring.
Startup or small-team experience and broad ownership are valuable. Web scraping, entity resolution, data enrichment, LLM-based extraction, BigQuery, and Dataform experience are also useful.
Compensation & Benefits
Base salary: $80,000 to $120,000 USD annually. Visa sponsorship is not available.
Location
Remote, candidates must be based in Latin America.
Similar jobs
Search more Data Engineer jobs🕒 yesterday
Own and operate a production data platform end to end, designing multi-source ingestion, event-driven processing, and orchestrated workflows that deliver reliable, up-to-date data for downstream products.
🕒 2 days ago
Build and scale reliable data pipelines, establish dbt standards and quality controls, and create self-service tools and AI agents that help teams access and use data independently.
🕒 3 days ago
Scale the data platform by building reliable pipelines, improving data models and quality, and enabling lower-latency analytics while shaping engineering practices as the team grows.
CAS | Engenharia de Dados SR (Preferencialmente para mulheres)
🔥 20 hours ago
Sicredi👥 10,000+ employees🏢 Financial Services
🔥 20 hours ago
Engenheira de Dados Sênior responsável por integrar fontes, desenvolver e otimizar pipelines e data marts, garantir governança e qualidade dos dados e orientar tecnicamente a equipe.