Work Type
Remote
Location
United States, Canada, United Kingdom, Ireland, Belgium, Netherlands, Luxembourg, Switzerland, Austria, Denmark, Sweden, Norway, Finland, Iceland, Italy, Spain, Portugal, Australia, New Zealand, Saudi Arabia, Israel, Monaco
Compensation
$40 per task
About this opportunity
Overview
ExpertWoka is seeking Subject Matter Experts to support AI conversation evaluation research. In this contract assignment, you will review conversations between participants and AI models, assess whether enough context was gathered before recommendations were made, identify missed opportunities for clarification, and provide structured feedback to improve conversational quality.
Responsibilities
- Review AI conversations involving real-world interpersonal and everyday decision-making scenarios.
- Identify missing context and recommend relevant follow-up questions.
- Evaluate whether additional information would have changed the AI’s recommendations.
- Distinguish between critical context and non-essential information.
- Conduct qualitative reviews and document findings using structured evaluation guidelines.
- Collaborate with research teams to ensure consistent, high-quality assessments.
Requirements
- 3–5 years of direct practice experience in a relevant professional field.
- Experience in at least one of the following areas: social work, counseling, therapy, psychology, qualitative research, HCI, communication, mediation, coaching, student services, organizational consulting, or related advisory work.
- Must be based in the United States, Canada, the United Kingdom, Ireland, Belgium, the Netherlands, Luxembourg, Switzerland, Austria, Denmark, Sweden, Norway, Finland, Iceland, Italy, Spain, Portugal, Australia, New Zealand, Saudi Arabia, Israel, or Monaco.
Qualifications
- Licensed Social Work background with MSW or equivalent graduate-level qualification, plus LCSW, LICSW, LMSW, or equivalent social work license/credential.
- Counseling, Therapy, or Psychology background with MA/MS and LMFT, LPC, LCPC, or equivalent counseling/therapy license or credential; or PhD/PsyD in Clinical, Counseling, Social, or Industrial-Organizational Psychology.
- Qualitative Research, HCI, or Communication background with MA/PhD or equivalent advanced expertise in HCI, Communication, Anthropology, Sociology, Qualitative Research, or a closely related discipline.
- Mediation, Coaching, Student Services, Organizational Consulting, or related advisory background with relevant professional qualification, credential, or demonstrated subject-matter expertise.
- Experience eliciting relevant context when clients initially provide incomplete information.
- Experience with counseling, therapy, psychological assessment, interviewing, case review, qualitative research, user research, conversation analysis, thematic analysis, qualitative coding, or similar activities.
Schedule
- Full-time commitment required: 40 hours per week.
- Estimated project length: 3 weeks.
- Estimated start date: October 1.
Project Details
- Contractor assignment.
- Pay-per-task compensation.
- Approximate average handling time: 75 minutes per task.
