About this opportunity
Overview
Prolific is hiring a Director of AI Research to lead a small, high-caliber research team working on evaluation methods, benchmarks, environments, and datasets that support frontier AI development. The role sits at the intersection of human data, model evaluation, safety, and human-AI interaction, with close collaboration across product, engineering, executive leadership, frontier labs, regulators, and the research community.
This position is remote, but candidates must be based within commutable distance of at least one of Prolific’s global hubs: London, New York, or San Francisco. The base salary range disclosed for this full-time role in San Francisco or New York is $275,000 to $400,000 annually.
What You’ll Work On
- Improving model capability through expert data, including determining which experts, tasks, and levels of signal add the most value.
- Building agents and environments for training and evaluation, with oversight that can scale.
- Measuring safety and human impact topics such as persuasion, perception, and reliance using real, representative participants.
- Identifying what actually moves a model by measuring which data, experts, and tasks drive capability gains.
- Creating trustworthy measurement systems, including evaluations, validated judges, and human baselines resilient to saturation and contamination.
- Designing tasks and data that improve signal quality, difficulty, and coverage.
Responsibilities
- Own the research strategy for work that advances more capable models, safer models, and evidence the field can trust.
- Set a high scientific bar for methods and results that withstand peer review and scrutiny from frontier labs.
- Turn research into reusable methods, benchmarks, tools, and new products that Prolific can build on.
- Build a portfolio of public benchmarks, environments, and datasets that become reference points for frontier labs.
- Partner with product leaders across agents, evaluation, multimodal, safety, and specialist domain work.
- Represent Prolific’s science with regulators, safety institutes, academia, and frontier lab research leaders.
- Work with product and commercial teams to convert research output into offerings customers adopt and return for.
- Report research impact to the executive team and board in terms of customers, revenue, and scientific standing.
- Lead, hire, and grow a UK/US team of research engineers, research scientists, and forward deployed researchers.
- Raise the bar through hiring and help senior researchers stay close to the hardest problems while leading their own agendas.
Requirements
- Credibility in post-training data, RL environments, AI evaluation, or safety and human-AI interaction, backed by papers, benchmarks, or methods used by others.
- Strong grounding in experimental design and measurement with human participants, including sampling, validity, reliability, and bias.
- 8+ years in research experience, including 3+ years leading researchers across locations and growing senior people.
- A track record of turning research into outcomes customers use, such as benchmarks, datasets, environments, or tools adopted by others.
- A clear point of view on where frontier AI data and evaluation are headed, plus the judgment to make hard portfolio decisions.
- Ability to make the case for those decisions to executives, customers, and the research community.
- Hands-on technical depth to review methods, read code, and challenge results directly.
Nice to Have
- Experience at a frontier lab, AI safety institute, or human data/evals company.
- Background spanning machine learning and behavioural or social science such as psychology, psychometrics, economics, or computational social science.
- Experience building agentic evaluations or LLM-as-judge pipelines at scale.
- Experience working with regulators or standards bodies on AI evaluation.