Software Engineer
Fivetran👥 10,000+ employees🏢 Software Development🤝 B2B
Design, implement, and maintain core data pipelines to ensure reliable, secure, and scalable data movement while collaborating with cross-functional teams.
Data Infra Engineer
🕒 yesterday
C++SQLDistributed SystemsData Engineering
📜 Description
- Design, develop, and maintain high-throughput distributed backend components and services (C++, Flume/Apache Spark/Flink/Beam) powering Waymo's Commercialization Data Lake.
- Own the scalable ingestion of data from raw system telemetry and RPC logs, processing petabytes of logs into reliable, daily canonical base tables.
- Build, harden, and democratize Data Pipeline Services, offering standard templates and self-service gateways for downstream consumption.
- Implement foundational compliance, privacy, and security infrastructure (e.g., GDPR wipeout, ACL management, BCID enforcement, and identity proxies).
- Establish rigorous operational standards, guarding pipelines with regression tests, staging environments, Canary/Progressive rollouts to defend high SLO.
- Collaborate with Data Engineers and Product Data Scientists to set global platform consistency, dependency tracing, and linear DAG architecture guidelines.
🛠️ Requirements
- BS/MS in Computer Science, Computer Engineering, or equivalent.
- 4+ years of professional backend software engineering experience, focused heavily on distributed systems, databases, and production monitoring.
- Excellent understanding of data structures, algorithms, and fundamental software design principles.
- Advanced proficiency in C++ and SQL applied to high-throughput batch and streaming processing pipelines.
- Prior experience running data pipelines through hardened release setups (Canarying, Staging environments, Automated Rollbacks).
- Extensive experience with distributed processing frameworks (e.g., Spark, Flink, Beam, Hadoop, Flume, or equivalent large-scale processing engines).
- Experience in building and maintaining Data Warehouse / Data Lake systems.
- Experience baking Data Governance, Privacy, or Compliance layers (e.g., encryption, GDPR, RBAC/ACLs) directly into core infrastructure.
Similar jobs
Search more Data Engineer jobs
Software Engineer, Security Rules
Cloudflare👥 10,000+ employees🏢 Computer And Network Security🤝 B2B
As a Systems Engineer on the Security Rules team, you will design, implement, and maintain software systems for Cloudflare's Application Security offerings, ensuring reliability and scalability.
Software Engineer, Network Performance & Reliability (Argo)
Cloudflare👥 10,000+ employees🏢 Computer And Network Security🤝 B2B
As a Software Engineer on the Argo team, you'll enhance network connectivity for Cloudflare's products, collaborating across engineering teams and utilizing cutting-edge technologies.
Software Engineer, Cloudflare Network Interconnect
Cloudflare👥 10,000+ employees🏢 Computer And Network Security🤝 B2B
You will build and extend systems that provision, monitor, and orchestrate private network interconnects at scale for Cloudflare's enterprise customers.
Software Engineer
Fivetran👥 10,000+ employees🏢 Software Development🤝 B2B
Join Fivetran as a Software Engineer to develop enterprise-grade software and enhance data integration systems, impacting the product's growth and customer success.