Role Description
We are looking for a skilled Data Engineer with strong expertise in Python and Microsoft Azure to design, build, and maintain scalable data pipelines and infrastructure. You will work closely with data scientists, analysts, and business stakeholders to ensure high-quality data availability and performance across platforms.
Responsibilities
- Design, develop, and optimize scalable data pipelines using Python and Azure Data Services.
- Build and manage data workflows using Azure Data Factory, Databricks, and Azure Synapse Analytics.
- Develop and maintain ETL/ELT processes to support analytics and reporting.
- Collaborate with data scientists and analysts to support data needs and enable actionable insights.
- Ensure data integrity, governance, and security across all platforms.
- Monitor and troubleshoot data pipeline performance and reliability.
- Write clean, testable, and efficient code following best practices.
- Participate in code reviews and documentation.
Profile
- Bachelor's degree in Computer Science, Engineering, or a related field.
- 5+ years of experience in data engineering or a similar role.
- Strong proficiency in Python for data manipulation and pipeline development.
- Hands-on experience with Azure Data Services (e.g., Azure Data Factory, Azure Databricks, Azure Synapse, Azure Blob Storage).
- Experience with SQL and relational/non-relational databases (e.g., SQL Server, Cosmos DB).
- Familiarity with data modeling, warehousing, and performance tuning.
- Understanding of CI/CD pipelines and version control (e.g., Git).
- Excellent problem-solving and communication skills.
- Experience with big data tools such as Spark or Kafka.
- Knowledge of DevOps practices and tools (e.g., Azure DevOps).
- Exposure to machine learning workflows or MLOps.
- Certification in Microsoft Azure (e.g., Azure Data Engineer Associate).
