Senior Azure Data Engineer - Mindera

At Mindera, we build technology products that create meaningful business impact. We work with modern cloud technologies and encourage engineering ownership, collaboration, continuous learning, and self-organization.

Our Data Engineering teams build scalable, reliable, and high-performance data platforms that enable analytics, reporting, machine learning, and data-driven decision-making.

Requirements

What We're Looking For

Design, develop, and maintain scalable data pipelines and ETL/ELT workflows using Azure Databricks
Build data processing solutions using Apache Spark and PySpark for large-scale datasets
Develop and optimize batch and real-time/streaming data pipelines
Implement modern Medallion Architecture using Bronze, Silver, and Gold data layers
Work extensively with Delta Lake for reliable, scalable data storage and processing
Develop complex and optimized SQL queries for data transformation and analytics
Integrate data from multiple sources including databases, APIs, files, cloud storage, and streaming platforms
Work with Azure services such as ADLS Gen2, Azure Data Factory, Azure Key Vault, Azure DevOps, and Azure Monitor
Optimize Spark workloads through effective partitioning, caching, joins, cluster configuration, and file-format optimization
Implement data quality, validation, monitoring, logging, and observability across data pipelines
Apply appropriate data modelling and dimensional modelling practices for analytics workloads
Implement CI/CD, source control, automated testing, and deployment practices for data engineering solutions
Contribute to architecture and technology decisions for modern cloud data platforms
Troubleshoot production data pipelines and proactively improve reliability, performance, and cost efficiency
Collaborate with Data Scientists, Data Analysts, Software Engineers, Product Managers, and business stakeholders
Mentor junior and mid-level engineers and contribute to engineering best practices
Create technical documentation covering data pipelines, architecture, processes, and operational procedures

Must-Have Skills

7+ years of professional experience in Data Engineering
Strong hands-on experience with Azure Databricks
Excellent knowledge of Apache Spark / PySpark
Strong programming experience in Python
Advanced SQL skills
Hands-on experience building scalable ETL/ELT data pipelines
Strong understanding of Delta Lake and Medallion Architecture
Experience with Azure Data Lake Storage Gen2 (ADLS)
Experience with Azure Data Factory or equivalent orchestration tools
Strong understanding of data warehousing, data lakes, and data modelling
Experience with batch processing and preferably streaming data processing
Hands-on experience with Git and CI/CD practices
Experience working in Agile/Scrum or similar engineering environments
Strong analytical, problem-solving, and troubleshooting skills
Excellent communication and stakeholder-management skills

Good-to-Have Skills

Experience with Unity Catalog and Databricks governance
Experience with Azure Event Hubs, Kafka, or Spark Structured Streaming
Experience with dbt or similar transformation frameworks
Experience with Terraform / Infrastructure as Code
Knowledge of Azure Synapse Analytics
Experience with data quality and observability frameworks
Understanding of security, access control, lineage, and governance
Experience with Databricks Workflows / Jobs and automated deployment
Exposure to machine learning or AI data pipelines
Experience optimizing cloud infrastructure and data-processing costs

What We Look For

Strong ownership and ability to work independently
A problem-solving mindset with attention to engineering quality
Ability to design solutions rather than simply implement requirements
Clear communication with technical and non-technical stakeholders
Willingness to mentor others and share knowledge
Curiosity and enthusiasm for learning new technologies
Ability to work effectively in a collaborative, self-organized engineering environment

Nice to Have

Experience working with feature stores and ML pipelines
Exposure to streaming technologies (Kafka, Kinesis, etc.)
Understanding of data governance and data quality frameworks
Experience with LLM-based applications

Benefits

We offer

Flexible working hours (self-managed)
Annual bonus, subject to company performance
Access to Udemy online training and opportunities to learn and grow within the role

At Mindera we use technology to build products we are proud of, with people we love.

Software Engineering Applications, including Web and Mobile, are at the core of what we do at Mindera.

We partner with our clients, to understand their products and deliver high-performance, resilient and scalable software systems that create an impact on their users and businesses across the world.

You get to work with a bunch of great people, and the whole team owns the project together.

Our culture reflects our lean and self-organisation attitude.

We encourage our colleagues to take risks, make decisions, work in a collaborative way and talk to everyone to enhance communication. We are proud of our work and we love to learn all and everything while navigating through an Agile, Lean and collaborative environment.

Our offices are located: Porto, Portugal | Aveiro, Portugal | Coimbra, Portugal | Leicester, UK | San Diego, USA | Chennai, India | Bengaluru, India

Senior Azure Data Engineer - Databricks