Python AI Engineer

Home - Careers - Python AI Engineer

Python AI Engineer

Home - Careers - Python AI Engineer

Python AI Engineer

Learn more about us

LOCATION: Hanoi, HCMC

Job Description

We are looking for a Python AI Engineer to join us in building and advancing our AI Indexing & Search Pipeline. In this role, you will be responsible for designing and implementing end-to-end pipelines for data crawling, content processing, vector embedding generation, and indexing to power high-performance semantic search capabilities.

• Pipeline Development & Core Architecture:
• Develop, optimize, and maintain the Python-based Batch AI Indexer system.
• Build and operate an end-to-end data pipeline
• Build and optimize Semantic Search / Vector Search capabilities.
• Data Crawling & Processing:
• Crawl and collect content from diverse web sources (Homepage, Naver Blog, and other relevant web data sources).
• Optimize text chunking strategies and data preprocessing tailored for language models.
• AI Models & Search Engine Integration:
• Utilize OpenAI Embeddings for content vectorization.
• Integrate and manage Elasticsearch 8.x for vector storage and search (working extensively with 1536-dimensional dense_vector and the Korean Nori Analyzer).
• Utilize FAISS for local vector search experiments and rapid prototyping.
• Batch, State & Reliability Management:
• Design and implement mechanisms for Incremental Indexing and Full Re-indexing.
• Manage incremental batch states using Redis.
• Track batch execution history and metadata using PostgreSQL.
• Implement robust retry mechanisms and error-handling logic to ensure high system reliability.
• Scheduling, Deployment & Operations:
• Configure automated job scheduling using APScheduler or Linux Crontab.
• Deploy and operate batch applications on Linux environments using systemd and Python virtual environments (venv).
• Perform logging, monitoring, and troubleshooting for batch job operations.
• Collaborate closely with Backend, AI, and DevOps team members to deploy and scale system operations.
  

Required Skills And Experience

• Core Experience & Technical Skills:
• 2–4+ years of strong hands-on development experience in Python.
• Proficiency in web scraping and crawling libraries (e.g., BeautifulSoup, Scrapy, Playwright, or Selenium).
• Practical experience with Elasticsearch 8.x (specifically Vector Search, dense_vector, kNN search, and custom analyzer configurations like Nori).
• Deep understanding and hands-on experience with Embedding models (e.g., OpenAI Embeddings, HuggingFace) and text chunking/preprocessing techniques.
• Databases & System Architecture:
• Solid skills in Redis (for caching/state management) and PostgreSQL (or equivalent relational databases).
• Strong knowledge of Batch Processing architectures, state management, idempotency, and error handling within data pipelines.
• Experience with FAISS or other vector search libraries for local experimentation.
• DevOps & Environment:
• Proficient in Linux environments, application deployment via systemd, and Python virtual environments.
• Experience with scheduling tools (crontab, APScheduler).
• Strong command of Git, logging frameworks, and system troubleshooting.
• Nice-to-Have Qualifications:
• Experience with Korean Natural Language Processing (Korean NLP).
• Experience with LLM orchestration frameworks such as LangChain or LlamaIndex.
• Hands-on experience with Docker, Kubernetes, or CI/CD pipelines.
  

Benefits

• Competitive and negotiable salary (UP TO 50M)
• Probation salary is 100% of official salary
• 13th-month salary and performance review twice a year
• Bonus for special occasions each year (Labor Day, National Day, Solar New year, Lunar New Year)
• Project bonus
• Employee's professional certification and training allowances subject to company regulations
• Opportunity to participate in internal training programs on AI, foreign languages, and professional development
• Health Care Insurance
• Annual Health Assessment
• Social, health and unemployment insurance following government policy
• Enjoy company summer trips and other team building activities held monthly and quarterly
• Professional, creative and dynamic working environment
• Work five days per week with flexible check-in time
  
  

CONTACT

• Send CV to: [Confidential Information]
• Hotline: +842466666100 (Recruitment Team)
• Facebook GEM Careers: https://www.facebook.com/tuyendungGEM
• Linkedin GEM Careers: https://www.linkedin.com/company/gem-careers-hiring/