AI Native Data Engineer

Quick answer: what is this job?

PALO IT is hiring a Data Engineer in Sao Paulo to design and maintain scalable data solutions that transform raw data into actionable insights for a leading insurance company. The role involves working with modern data technologies and frameworks to support a data-driven decision-making culture.

Who We Are

Building the AI-first frontier enterprise.

We are a global technology consultancy with a trademarked, AI-first approach, Gen-e2™. We redefine how enterprises build digital products and transform their organizations with AI. We're proud to be a World Economic Forum New Champion, and a B Corp-certified company.

• We are small enough to care locally, big enough to deliver globally (9 countries, 400+ experts from 50+ nationalities)

• We are becoming an agentic organization, adopting the AI-native operating model we bring to our clients.

• We are robust and resilient (100% independent, 0 debt, founded 2009)

• We are AI-native professionals who invest in what we believe and work as a collective intelligence

• We are positive, courageous and deliver at the leading edge

What does a Data Engineer do at PALO IT?

As a Data Engineer at PALO IT, you will:

• Design, build, and maintain scalable and secure data pipelines within cloud environments (AWS, GCP, or Azure) to ensure seamless data flow.

• Integrate large volumes of structured and unstructured data from core insurance systems, including policies, claims, CRM, and ERP.

• Automate data ingestion, transformation, and quality processes utilizing tools such as Apache Spark, dbt, Kafka, and Airflow.

• Build and maintain modern data lakes and warehouses, leveraging solutions like Snowflake, BigQuery, and Redshift.

• Implement data quality, lineage, and governance validations using tools such as Great Expectations, Deequ, and OpenMetadata.

• Ensure compliance with data regulations (GDPR, SOC2, etc.) and internal security policies to protect sensitive information.

• Design optimized datasets that facilitate machine learning models, predictive analytics, and visualization in tools like Power BI, Looker, and Tableau.

• Collaborate with data scientists, architects, and business stakeholders to democratize access to trusted data across the organization.

• Document data architectures, pipelines, transformation standards, and lineage for transparency and reproducibility.

What experience do you need for this role?

You're a strong fit for this Data Engineer role if you have:

Required / Must-have:

• Experience with modern data processing frameworks such as Spark, Databricks, dbt, Kafka, and Apache Beam.

• Strong command of Python and advanced SQL for data manipulation and processing.

• Hands-on experience with Azure cloud platforms, particularly Data Factory and Synapse.

• Knowledge of modern Data Lake and Data Warehouse design principles (Snowflake, Redshift, BigQuery).

• Familiarity with DataOps practices, testing, and CI/CD implementation in data pipelines.

• Experience with workflow orchestration systems like Airflow or Dagster.

• Understanding of data quality and governance frameworks including Great Expectations, Deequ, DataHub, and OpenLineage.

• Upper-intermediate English level (B2 or higher) for effective global collaboration.

Nice-to-have:

• Knowledge of streaming technologies, such as Kafka, Pub/Sub, and Kinesis.

• Familiarity with event-driven architectures and microservices.

• Previous experience in the financial or insurance sector.

• Relevant certifications such as Azure Data Engineer or Fabric Data Engineer.

• Experience with MLOps solutions like Vertex AI, SageMaker, or MLFlow.

More About PALO IT

Why join PALO IT? You'll enter a forward-thinking, collaborative environment among a passionate international team that pushes the boundaries of AI-native development. You'll have continuous personal and professional growth through our PALO IT Academy, specialized training, and R&D projects.

Global opportunities for international mobility and collaboration with visionary clients and frontier tech partners are available. You'll be empowered to innovate and shape the implementation of next-level AI methodologies as part of a collective intelligence. We offer competitive compensation and benefits that reflect your skills, impact, and leadership.

Diversity, Equity, and Inclusion

As a B Corp, PALO IT provides equal opportunities and assesses candidates solely on skills, potential, and motivation. PALO IT welcomes applications regardless of gender identity or expression, sexual orientation, age, background, ethnicity, religion, disability, or any other protected attribute.

Frequently Asked Questions about this role

Q: What is the work model for this role?
A: Hybrid

Q: What technologies will I be working with?
A: You will work with AWS, Azure, GCP, Apache Spark, and a variety of data management tools and frameworks.

Q: Is there a focus on language skills for the role?
A: Yes, an upper-intermediate English level (B2 or higher) is required for collaboration.