Senior Data Engineer (60 DE 40 BE)

Main Tech Stack: Node.js / NestJS, Python / Airflow, SQL / dbt, Snowflake, PostgreSQL, Kafka

Scope of Work:

• Backend / Data platform focused. No end-user UI. Batch-based processing.
• Build a batch data pipeline to ingest, validate, transform, and serve enrollment data from multiple sources.

Key Responsibilities:

• Data ingestion: Design, build, and maintain scalable batch data pipelines to ingest enrollment data from disparate sources using Python, Airflow, and SQL.
• Data quality & validation: Implement robust data validation rules and data quality frameworks to ensure accuracy, consistency, and reliability across all datasets.
• Data transformation: Perform efficient batch data transformations and historical tracking using dbt, Snowflake, and PostgreSQL.
• Historical tracking
• REST API development: Build high-performance RESTful APIs using Node.js / NestJS to serve processed enrollment data to downstream systems.
• Event publishing: Integrate asynchronous event publishing/messaging workflows using Apache Kafka.

• Experience: 5+ years of experience as a Data Engineer, Backend Engineer, or Data Platform Engineer.
• Core Tech Stack:
• Backend: Strong proficiency in Node.js / NestJS.
• Data & Orchestration: Solid experience with Python, Apache Airflow, and dbt.
• Databases & Warehousing: Hands-on experience with Snowflake, PostgreSQL, and advanced SQL.
• Messaging/Events: Experience with Apache Kafka or similar messaging systems.
• Domain & Concept Knowledge: Strong understanding of batch processing architecture, data modeling, historical data tracking (SCD), and REST API design.