AI Quality Analyst – Japanese (Personalization)

Job Description

Turing is hiring Japanese-speaking AI Quality Analysts to evaluate the quality of personalized AI responses.

The project focuses on testing a personalization feature for Gemini, assessing how effectively the model uses information from sources such as previous Gemini conversations and connected Google activity to produce relevant, accurate, and helpful responses.

You’ll create realistic multi-turn prompts, compare AI responses side-by-side, identify personalization problems, and provide detailed feedback to improve model quality.

What You’ll Do

• Design multi-turn conversational prompts in Japanese based on realistic personal contexts.
• Evaluate whether AI responses use personalization appropriately.
• Identify unsupported assumptions, hallucinations, and incorrect personalization.
• Assess responses for grounding, integration, naturalness, and helpfulness.
• Compare and rank two AI responses side-by-side.
• Identify subtle differences in response quality and reasoning.
• Write clear rationales explaining your rankings and evaluation decisions.
• Provide detailed annotations and constructive feedback.
• Verify whether appropriate data sources and conversation context were used.
• Follow required data-hygiene procedures during evaluations.

Requirements

• Strong Japanese reading and writing proficiency.
• Excellent analytical and critical-thinking skills.
• Ability to identify subtle problems in AI-generated responses.
• Strong written communication and attention to detail.
• Comfortable designing creative prompts and multi-turn conversations.
• Experience with AI evaluation, data annotation, content moderation, or similar work is strongly preferred.
• BA/BS degree or equivalent relevant experience in areas such as Linguistics, Journalism, Policy, Law, Ethics, Computer Science, or another analytical discipline.
• Laptop/desktop computer and reliable internet connection.
• Willingness to use a primary personal Google account and enable required personal data sources for genuine personalization testing.
• Ability to work independently in a remote environment.

Schedule

Candidates must be available for at least 4 hours per day and up to 20 hours per week, including 4 hours of overlap with Pacific Time (PST).

Application Process

Shortlisted candidates will:

1. Receive a Job Interest Form.
2. Complete an assessment following profile review.
3. Complete the assessment within 24 hours of receiving it.
4. Proceed to pre-onboarding requirements if successful.