Senior Data Engineer

Job Summary:

We are seeking an experienced Data Engineer to join our dynamic team. The ideal candidate will have a solid background in data engineering, with expertise in Python, ETL tools, Airflow, and Hadoop. You will be responsible for building, maintaining, and optimizing data pipelines that handle high-volume transactions exceeding 50 million per day.

Key Responsibilities:

• Design, develop, and maintain scalable data pipelines and data models for large transaction systems.
• Utilize Python and ETL tools to extract, transform, and load data efficiently from various sources.
• Implement and manage workflows using Apache Airflow to automate data processing tasks.
• Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and ensure data quality and availability.
• Optimize data processing performance and reliability in a Hadoop environment.
• Monitor and troubleshoot data pipeline issues to ensure operational efficiency and reliability.
• Document processes, data flow, and architecture for future reference and compliance.
• Stay current with industry trends and emerging technologies in data engineering.

Required Qualifications:

• 5+ years of experience in data engineering or a similar role.
• Strong proficiency in Python for data processing and manipulation.
• Hands-on experience with ETL tools (e.g., Talend, Informatica, Apache NiFi).
• In-depth knowledge of Apache Airflow for workflow management.
• Experience with big data technologies such as Hadoop and its ecosystem (e.g., HDFS, Hive, Spark).
• Proven experience in building and maintaining big transaction systems (handling over 50 million transactions per day).
• Familiarity with cloud platforms (e.g., AWS, Azure, GCP) is a plus.

Compensation & Benefits:

• Competitive Salary
• Supportive and flexible working environment
• MacBook provided
• Premium health insurance
• SHUI contributions which comply with local laws along with premium health insurance
• Guaranteed 13th-month salary
• 15 paid annual leave days per year
• Work from home 3 days/month