AI Quality Analyst – Personalization Evaluation

Job Description

Turing is hiring AI Quality Analysts to evaluate a new personalization feature for Gemini.

You’ll test how effectively the AI uses information from your past Gemini conversations, Gmail, Google Search, and YouTube activity to provide more relevant and helpful responses.

The work combines creative prompt design with detailed AI evaluation, including comparing responses, checking whether personalization is accurate, and writing clear rationales explaining your ratings.

What You’ll Do

• Design multi-turn conversational prompts, typically spanning 1–5 turns.
• Create prompts based on your own personal context and experiences.
• Evaluate whether AI responses use personalization appropriately.
• Check whether personalized claims are properly grounded in available information.
• Identify hallucinations, incorrect assumptions, poor inferences, and forced personalization.
• Evaluate how naturally personal information is integrated into responses.
• Compare two AI responses side-by-side and rank them for overall quality and helpfulness.
• Write clear, evidence-based rationales explaining your evaluations.
• Review debug information to verify whether relevant data sources were properly used.
• Follow required data-hygiene procedures for evaluation conversations.

Requirements

• Strong English reading and writing skills.
• Exceptional analytical thinking and attention to detail.
• Ability to evaluate nuanced or ambiguous AI responses.
• Strong written communication and ability to provide structured evaluation rationales.
• Comfortable designing creative, multi-turn prompts.
• Ability to work independently in a remote environment.
• Desktop or laptop with reliable internet access.
• Full-time schedule flexibility within your local time zone.

Personal Google Account Requirement

Candidates must be willing to use their primary personal Google account rather than a testing account and enable relevant personal data sources for the evaluation.

These sources may include:

• Gemini conversation history
• Gmail
• Google Search activity
• YouTube activity

Applicants should consider this requirement carefully before applying.

Education & Experience

• BA/BS degree or equivalent relevant experience.
• Relevant backgrounds may include Linguistics, Journalism, Policy, Law, Ethics, Computer Science, or other analytical fields.
• Previous experience in AI quality evaluation, data annotation, content moderation, or similar work is strongly preferred.

Evaluation Process

1. Shortlisted applicants receive a Job Interest Form.
2. Profiles are reviewed.
3. Selected candidates receive an assessment that must be completed within 24 hours.
4. Successful candidates are contacted regarding pre-onboarding requirements.