About the Role
We are seeking a Senior Data Engineer to build and stabilize the next generation of Cake Digital Bank's unified data infrastructure — powering analytics, dashboards, and real-time intelligence across the organization.
You will work across streaming, batch, and lakehouse layers, ensuring that our data pipelines and platforms — BigQuery, Doris, and Iceberg — operate with reliability, observability, and cost-efficiency.
This role is central to the evolution of Cake's data mesh and lakehouse architecture, where dashboards, dbt transformations, and machine-learning features all depend on a stable, scalable, and auditable data foundation.
Key Responsibilities
1. Data Infrastructure Stability & Reliability
- Design, implement, and maintain high-availability data pipelines that replicate data from OLTP systems to analytical warehouses (BigQuery, Doris)
- Build resilience and recovery patterns — checkpointing, replay queues, schema-aware ingestion, deduplication, and versioned storage.
- Lead incident response, RCA, and SLA management for data infrastructure components.
- Implement end-to-end observability across ingestion, transformation, and serving layers .
2. Unified Storage & Lakehouse Evolution
- Architect and operationalize Apache Iceberg as the central data storage layer to unify data across BigQuery and Doris.
- Define data layout, partitioning, compaction, and schema evolution strategy for Iceberg tables stored on GCS.
- Design cross-system metadata synchronization between Iceberg catalog, BigQuery external tables, and Doris engines.
3. Platform Automation & Scalability
- Develop and maintain infrastructure-as-code (Terraform, Helm, Config Sync) for Airflow, Flink, Doris, and BigQuery resources.
- Build self-service templates for creating new pipelines, enabling consistent deployment and monitoring.
- Optimize data infrastructure configurations to support workload growth while controlling costs.
- Automate schema detection, dependency validation, and backfill workflows for safe and predictable releases.
4. Data Governance, Quality & Security
- Integrate with DataHub to maintain end-to-end lineage, glossary, and policy-tag enforcement.
- Apply column-level security and row-level policies for sensitive and restricted data (PII, PCI, financial metrics)in BigQuery and Doris.
- Establish validation checks and data contracts to ensure quality and consistency across streaming and batch paths.
- Collaborate with Compliance and Risk teams to maintain auditable data flow and access traceability.
5. Performance Optimization & Cost Efficiency
- Continuously tune BigQuery slot usage, query design, and reservation policies by domain.
- Improve Doris query performance through partition pruning, tablet balancing, and index tuning.
- Design caching, pre-aggregation, and materialized-view strategies to accelerate dashboards while reducing query cost.
- Track job efficiency, data duplication, and storage growth across the platform.
6. Collaboration & Leadership
- Partner with BI, Risk, and ML teams to deliver reliable, low-latency data products that power business decisions.
- Mentor junior engineers on best practices for streaming systems, Airflow orchestration, and infra automation.
- Participate in architecture design sessions to guide the evolution of Cake's multi-engine, unified lakehouse platform.
Required Skills & Experience
- 5+ years of hands-on experience in data engineering or data platform development in a cloud-native environment
- Strong proficiency in SQL (BigQuery, Doris, or equivalent MPP engines) and Python for building, monitoring, and automating data pipelines.
- Proven experience operating streaming and CDC systems such as Dataflow, Flink, Debezium, or Datastream, with solid understanding of checkpointing, offset management, and backpressure handling.
- Deep understanding of data modeling and warehouse optimization — partitioning, clustering, materialized views, caching, and query tuning.
- Hands-on experience implementing and maintaining infrastructure-as-code using Terraform, Helm, or Config Sync.
- Familiarity with Apache Iceberg (or Delta/Hudi) and lakehouse design patterns, including schema evolution and compaction.
- Strong background in observability and reliability engineering — building dashboards, alerts, and auto-remediation for data pipelines using Prometheus, Grafana, or Cloud Monitoring.
- Understanding of data governance and security concepts: column-level security, policy tags, lineage, and data classification using DataHub or similar systems.
- Working knowledge of orchestration and workflow automation tools such as Airflow or Dagster.
Our benefits:
- Competitive compensation including a 13th-month wage and up to 3 months of performance-based bonus.
- Macbook and essential equipment are provided.
- BE Corp budget (vary from your level) is allocated for using services such as transportation, food, and passenger car bookings in Be application.
- Annual health checks and premium medical healthcare (PTI) after probation.
- 15 days of annual leave is applied for the entire employees.
- Company trips, team-building activities, and happy hour events are organized on a quarterly or annual basis.
