AI Benchmark & Red-Teaming Specialist

Overview

Turing is seeking an analytical researcher and red-teaming specialist to build evidence-based web-research benchmark tasks that expose weaknesses in state-of-the-art AI models. The work centers on finding hard-to-surface facts, designing precise research tasks around them, testing those tasks against leading LLMs, and diagnosing failure modes in model search and reasoning.

This role is intended to support AI evaluation frameworks and help ensure frontier models are assessed against rigorous, non-trivial real-world research scenarios.

What You'll Do

• Research niche and difficult-to-find topics to uncover factual ground truths that are not easily surfaced by straightforward searches.
• Verify findings using primary or highly credible sources, cross-check conflicting evidence, and document exact source URLs.
• Write clear, answerable benchmark prompts with precise answer keys grounded in primary sources.
• Design benchmark tasks specifically aimed at revealing structural weaknesses in model research strategies.
• Run tasks against designated frontier models and confirm whether they trigger the intended failures.
• Review model reasoning, search trajectories, logs, and intermediate outputs to identify where and why failures occur.
• Distinguish between genuine model limitations and ambiguity in the prompt or task design.
• Apply validated research patterns across new domains and review tasks authored by other trainers when needed.

Requirements

• 1+ years of experience in investigative research, fact-checking, AI benchmark task authoring, red-teaming, or technical QA review.
• Experience authoring, testing, or reviewing tasks for AI benchmarks such as GAIA, BrowseComp, SimpleQA, SWE-bench, Terminal-Bench, or Humanity’s Last Exam (HLE).
• Intermediate to advanced web search and OSINT skills, including the ability to work with academic archives, local records, or obscure databases.
• Strong analytical ability to inspect reasoning chains, search logs, and source selection to diagnose root causes of failure.
• Ability to write clear, unambiguous benchmark prompts with reliable, primary-sourced answer keys.
• A bachelor’s degree or higher in a relevant engineering domain is required.

Preferred Qualifications

• Prior experience designing prompt-injection or reasoning-trap tasks that break state-of-the-art models.
• Advanced OSINT capabilities or published investigative or academic research demonstrating deep source verification.
• Niche domain expertise in areas with hard-to-find records, such as local government filings, regional sports history, legal archives, or historical patents.
• Experience as a lead reviewer, benchmark auditor, or quality owner on AI evaluation projects.

Engagement Details

• Up to 40 hours per week.
• 4 hours of overlap with PST.
• Contractor engagement.
• Engagement length: up to 4 weeks.
• Candidates must submit a Job Interest Form and assessment as part of the evaluation process.