The Data Engineer develops and maintains data pipelines using Python and SQL Server, supporting data integration and operations within a modern Azure-based data platform.
Data Engineer
π Description
- Designing and implementing robust data pipeline solutions within Azure Data Lake.
- Integrating and transforming data from various sources to enable comprehensive analytics and reporting.
- Ensuring data quality and integrity, implementing data governance practices.
- Collaborating with cross-functional teams to understand data requirements and develop scalable solutions.
- Monitoring and optimizing data processing workflows for performance and reliability.
- Documenting processes and architecture to ensure maintainability and scalability.
π οΈ Requirements
- Bachelor's degree in Computer Science, Information Technology, or a related field.
- A minimum of 5 years of experience working as a Data Engineer, with a strong emphasis on Azure Data Lake.
- Proficiency in Azure Data Services, particularly Azure Data Lake Storage, Azure Data Factory, and Azure Synapse Analytics.
- Demonstrated experience with data modeling, ETL processes, and data warehousing solutions.
- Strong programming skills in Python, SQL, and familiar with Java or C#.
- Excellent problem-solving skills and a detail-oriented approach to work.
- Ability to communicate technical concepts clearly to non-technical stakeholders.
- Strong organizational skills with the ability to manage multiple priorities and meet deadlines.
β¨ Benefits
- Paid Time Off
- Work From Home
- Health Insurance
- Training and Development
Full job description
We are seeking a highly skilled Data Engineer with extensive experience in Azure Data Lake to contribute to our dynamic team at Creative Chaos. The ideal candidate will be responsible for architecting, developing, and optimizing data pipelines that will facilitate the efficient processing of large datasets.
Responsibilities:
- Designing and implementing robust data pipeline solutions within Azure Data Lake.
- Integrating and transforming data from various sources to enable comprehensive analytics and reporting.
- Ensuring data quality and integrity, implementing data governance practices.
- Collaborating with cross-functional teams to understand data requirements and develop scalable solutions.
- Monitoring and optimizing data processing workflows for performance and reliability.
- Documenting processes and architecture to ensure maintainability and scalability.
- Staying abreast of new Azure technologies and best practices for data engineering.
Requirements
The successful candidate will possess the following qualifications:
- Bachelor's degree in Computer Science, Information Technology, or a related field.
- A minimum of 5 years of experience working as a Data Engineer, with a strong emphasis on Azure Data Lake.
- Proficiency in Azure Data Services, particularly Azure Data Lake Storage, Azure Data Factory, and Azure Synapse Analytics.
- Demonstrated experience with data modeling, ETL processes, and data warehousing solutions.
- Strong programming skills in Python, SQL, and familiar with Java or C#.
- Excellent problem-solving skills and a detail-oriented approach to work.
- Ability to communicate technical concepts clearly to non-technical stakeholders.
- Strong organizational skills with the ability to manage multiple priorities and meet deadlines.
Benefits
Perks & benefits
- Paid Time Off
- Work From Home
- Health Insurance
- OPD
- Training and Development
Similar jobs
Search more Data Engineer jobsWe are seeking an Azure Data Engineer with expertise in Databricks and Azure Data Factory to automate and optimize data integration processes.
As a Data Engineer - Spark Developer, you will design and improve data pipelines, utilizing Apache Spark and Azure services to create robust big data architectures.
As a Data Engineer, you'll design and implement big data solutions, focusing on batch and streaming ETL processes while collaborating with business analysis teams.
As a Data Engineer, you will own the data layer for credit risk decisions, ensuring data quality and availability for underwriting and renewal processes.
