Quality Evaluator – Fluent English (Europe & LATAM)

Overview

Anyone AI is looking for English-fluent experts to help evaluate and compare responses from advanced AI models. The role is remote, offered as an independent contractor opportunity, with a commitment of up to 3 months. The position is listed as part-time, and compensation is $7-$12 per hour.

What You'll Do

• Work with different AI models and tools across a broad range of prompts and topics.
• Compare two or more AI-generated responses side by side.
• Assess response accuracy, helpfulness, instruction-following, clarity, and overall quality.
• Follow detailed rating guidelines and multi-step evaluation workflows.
• Write clear, well-reasoned explanations for your evaluation decisions.
• Apply evaluation criteria consistently across tasks.

Requirements

• Fluency in English, including strong reading comprehension and writing skills.
• Extensive hands-on experience using LLMs and critically assessing their outputs.
• Strong analytical judgment and the ability to weigh multiple dimensions of response quality.
• Careful attention to detail and the ability to follow complex instructions.
• Ability to explain evaluation decisions clearly and convincingly.
• Comfort switching between AI models, tools, and multi-step workflows.

Nice to Have

• Previous experience in AI response evaluation, data annotation, or quality assurance.
• Experience in editing, research, fact-checking, or analytical writing.
• Familiarity with evaluation rubrics and side-by-side comparison tasks.