AI Agentic Evaluation Specialist Mandarin Chinese (Singapore)

Innodata (NASDAQ: INOD) is a leading data engineering company. With more than 2,000 customers and operations in 13 cities around the world, we are an AI technology solutions provider-of-choice for 4 out of 5 of the world's biggest technology companies, as well as leading companies across financial services, insurance, technology, law, and medicine.

By combining advanced machine learning and artificial intelligence (ML/AI) technologies, a global workforce of subject matter experts, and a high-security infrastructure, we're helping usher in the promise of AI. Innodata offers a powerful combination of both digital data solutions and easy-to-use, high-quality platforms.

Our global workforce includes over 5,000 employees in the United States, Canada, United Kingdom, the Philippines, India, Sri Lanka, Israel and Germany.

We are hiring a Singapore-based Mandarin Chinese (华语) language specialist to evaluate AI agents and large language model (LLM) outputs on real-world research and multi-step tasks.

This role involves reviewing the AI's full research workflow—not only the final answer. You will assess search queries, source traces, web research, tool calls, retrieved data, reasoning steps, citations, and Mandarin Chinese outputs to determine whether the model followed an accurate, safe, relevant, and instruction-compliant approach.

Key Responsibilities

• Review and evaluate AI/LLM agent workflows, including research steps, search strategy, tool usage, source selection, and final responses.
• Compare two AI-generated workflows and outputs to identify the stronger model response.
• Assess source credibility, factual accuracy, relevance, completeness, citation quality, and reasoning consistency.
• Identify incorrect information, unsupported claims, missing research, safety issues, cultural/contextual gaps, and instruction-following errors.
• Provide structured ratings and concise written feedback through the client platform.
• Evaluate Mandarin Chinese content for language quality, localisation, cultural relevance, and Singapore market context.

Required Skills

• Native or near-native proficiency in Singapore Mandarin Chinese (华语) and strong English comprehension.
• Currently based in Singapore, with familiarity with local culture, language usage, and market context.
• Experience or interest in AI evaluation, LLM evaluation, AI training, RLHF, data annotation, content rating, search relevance, search quality, research analysis, fact-checking, quality assurance, localisation, or linguistic evaluation.
• Strong online research, critical thinking, source verification, analytical, and written feedback skills.
• Ability to independently follow detailed guidelines and maintain accuracy across high-volume tasks.

Please note that the organization follows a strict no-fee recruitment policy and does not request any payment from candidates at any stage of the hiring process, including through third-party agencies or consultants. Candidates are advised to immediately report any such payment requests in connection with employment opportunities, as they are unauthorized and fraudulent.