Text Situated Alignment Specialist - High Cost

Overview

ExpertWoka is seeking Subject Matter Experts to support AI conversation evaluation research. In this short-term contractor assignment, you will review conversations between participants and AI models, assess whether sufficient context was gathered before recommendations were made, identify missed opportunities for clarification, and provide structured feedback to improve conversational quality.

Responsibilities

• Review AI conversations involving real-world interpersonal and everyday decision-making scenarios.
• Identify missing context and recommend relevant follow-up questions.
• Evaluate whether additional information would have changed the AI's recommendations.
• Distinguish between critical context and non-essential information.
• Conduct qualitative reviews and document findings using structured evaluation guidelines.
• Collaborate with research teams to support consistent, high-quality assessments.

Requirements

• 3–5 years of direct practice experience in a relevant field.
• Professional background in at least one of the following: social work, counseling/therapy, psychology, qualitative research, HCI, communication, mediation, coaching, student services, organizational consulting, or related advisory fields.
• Must be based in the United States, Canada, the United Kingdom, Ireland, Belgium, the Netherlands, Luxembourg, Switzerland, Austria, Denmark, Sweden, Norway, Finland, Iceland, Italy, Spain, Portugal, Australia, or New Zealand.
• Ability to work either 40 hours per week or 20 hours per week, with 4 hours of PST overlap.

Project Details

• Contractor assignment, short-term project.
• Estimated start date: October 1.
• Engagement length: 3 weeks.
• Compensation: Pay-per-task with a rate of $40 per task and an average handling time of 75 minutes.

Evaluation Process

• Profile review.
• ICF interview schedule.