A
Data Engineer (Databricks)
A
Data Engineer (Databricks)
aryan solutions pte. ltd.Early Applicant
- Posted a day ago
- Be among the first 10 applicants
Job Description
Required Technical Skills - Databricks
- Unity Catalog - catalogue/schema design, access control, and data lineage
- Lakeflow / Delta Live Tables for pipeline orchestration Delta Lake table format
- Databricks SQL and cluster/workspace administration (compute policies, pools, cost management)
- Databricks Asset Bundles (DABs) and Databricks Repos for CI/CD
- PySpark / Spark SQL for large-scale data transformation
- Working knowledge of Databricks system tables (audit logs, billing/usage, query history) for observability
- Minimum 5 years of hands-on experience in Data Engineering, Data Platform Engineering, or related disciplines.
- Minimum 3 years of hands-on experience with Databricks involving data pipeline development, platform administration, governance, and optimization.
Required Technical Skills - Common
- Strong SQL and Python (PySpark or general-purpose) for data engineering
- Data modeling - dimensional design, star/snowflake schemas, semantic layers
- CI/CD pipelines (e.g., Azure DevOps, GitHub Actions, GitLab CI) for data engineering workflows
- Infrastructure-as-code (Terraform preferred) for provisioning cloud data platform resources
- Hands-on experience on at least one hyperscaler - AWS, Azure, or Google Cloud
Understanding of data security and compliance frameworks applicable to government/public-sector environments
More Info
Key Skills
Databricks SQL
Infrastructure-as-code
Unity Catalog
Databricks Repos
GitLab CI
GitHub Actions
CI CD pipelines
Delta Lake
Lakeflow Delta Live Tables
About Company
Skills:
Spark SQL, Apache Flink, Pyspark, Apache Kafka, Databricks, Devops Tools, Adf, Sql, Airflow, Git workflows, AWS Kinesis, CI CD pipelines
Skills:
Pyspark, Databricks, Databricks SQL, dimensional modelling, incremental load patterns, data quality frameworks, Delta Lake, Git-based version control, Delta Live Tables, Databricks Workflows, Unity Catalog
Skills:
Docker, Terraform, Pyspark, Databricks, Sparksql, Sql, Python, Kubernetes, AWS, Airflow, dbt
Skills:
Data Modelling, Pyspark, Databricks, Python, Etl, ELT, data pipeline development, Databricks Workspace, Delta Lake, Lakehouse architecture, AI Agent
Skills:
Spark SQL, Data Modelling, Apache Flink, Agile Methodologies, Pyspark, Git, Apache Kafka, Databricks, Scrum, Airflow, Distributed Computing Architectures, DevOps practices, SQL Performance Optimization, CI CD, Azure Data Factory ADF, Databricks Notebooks Jobs, Batch and Real-Time Data Processing, Azure Data Platform, AWS Kinesis