Quality Analyst - Non US

Overview

ExpertWoka is sourcing detail-oriented Quality Analysts for a short-term, remote AI evaluation project.

This role involves reviewing and comparing responses generated by multiple chatbots, identifying quality issues, and providing clear final assessments based on defined guidelines.

Responsibilities

• Evaluate and compare responses generated by multiple chatbots.
• Assess responses for accuracy, relevance, completeness, clarity, and adherence to guidelines.
• Identify errors, inconsistencies, missing information, and quality issues.
• Provide structured feedback and final assessments for each evaluation task.
• Follow evaluation guidelines and maintain consistency across reviews.
• Document findings clearly and accurately.
• Collaborate with project and quality teams to resolve queries.
• Meet assigned productivity, quality, and turnaround-time targets.
• Escalate complex or ambiguous cases when necessary.

Requirements

• 3+ years of experience in Quality Assurance, data annotation, content evaluation, testing, or a similar role.
• Strong analytical, critical-thinking, and problem-solving skills.
• Excellent written and verbal communication skills.
• High attention to detail and ability to identify subtle errors.
• Ability to follow detailed instructions and evaluation guidelines.
• Comfortable working with AI tools, chatbots, and online platforms.
• Ability to work independently and manage multiple tasks.
• Prior experience with chatbot evaluation, LLM testing, or data-quality projects is an advantage.

Schedule

• Short-term contract: 16 weeks.
• Full-time availability required: 8 hours per day.
• Five working days per week, including weekends as required, with two weekdays off.
• 4 hours overlap with PST required.
• Start: Immediately.

Project Details

• Project-based role involving a defined number of evaluation tasks.
• Each task involves comparing multiple chatbots and providing a final assessment.

Evaluation Process

Application review, interest check form, and assessment link (general aptitude test).