Job Responsibilities
- Participate in the design, development, and operation of data processing pipelines (batch and streaming) on a Big Data platform.
- Collaborate in building and optimizing real-time data processing systems, ensuring high availability and performance.
- Develop and manage data orchestration workflows to ensure data is processed on time and can be effectively monitored.
- Implement data quality assurance, monitoring, and alerting to detect and respond to data pipeline issues.
- Collaborate on data modeling and query design to support analytics, reporting, and downstream systems.
- Implement and comply with data governance requirements, including access control, metadata management, and data lineage.
- Standardize and document data to support AI and Chatbot applications, ensuring data is used safely and in the correct business context.
- Work closely with cross-functional teams (Product, AI, BI, Infrastructure) to deliver data-driven projects.
- Share knowledge and mentor junior team members.
Requirements
- At least 3 years of experience working as a Data Engineer in large-scale data environments.
- Hands-on experience with Hadoop/Hortonworks (HDFS, YARN) in a production environment.
- Proven experience running Apache Spark in production (Spark SQL / DataFrame); experience with Structured Streaming is a strong advantage.
- Solid understanding of Kafka / Message Queues, including topics, partitions, consumer groups, offset/lag management, retry mechanisms, and Dead Letter Queues (DLQ).
- Strong advanced SQL skills on Hive, Impala, Presto, or Trino, including CTEs, window functions, and query optimization.
- Proficiency in Python for data processing (clean code, logging, pandas/numpy).
Preferred Qualifications
- Experience working with Big Data platforms or enterprise Data Platforms.
- Hands-on experience with real-time data processing systems.
- Knowledge of data governance and data security best practices.
- Experience participating in AI, Analytics, or Chatbot projects is a plus.
- Experience in technical leadership or team mentoring.
About Viettel Cyber Security (VCS)
With a proven track record of preventing more than 25,000 cyberattacks annually, managing over 1,000 websites and 16,000 applications, and serving 100+ customers including government ministries, large enterprises, and leading organizations in banking, finance, insurance, and securities, Viettel Cyber Security (VCS) is currently the No.1 cybersecurity service provider in Vietnam.
VCS is seeking a Data Engineer to join our team and play a key role in building, operating, and continuously improving our Big Data & Streaming platform, ensuring a stable, efficient, high-quality, and well-governed data ecosystem.
