Strategic Project Lead – Software Engineering

Overview

ExpertWoka is sourcing a Strategic Project Lead – Software Engineering for a short-term contractor engagement supporting software-engineering data and evaluation programs for frontier AI models.

This role is best suited to candidates with direct experience operating production-level SWE data programs and strong technical judgment. The work involves owning active programs end to end, including contributor sourcing and calibration, quality and throughput management, delivery recovery, and review of code, tests, and final artifacts.

What You'll Do

• Operate active software-engineering data and evaluation programs end to end.
• Manage contributor sourcing, calibration, onboarding, reviewer capacity, and performance.
• Track throughput, quality, rework, defects, cost, and delivery timelines.
• Review code, tests, task metadata, and final artifacts for technical correctness.
• Diagnose quality failures and improve task design, rubrics, instructions, and review processes.
• Lead recovery plans when programs fall behind on quality, throughput, cost, or timelines.
• Coordinate distributed software engineers, reviewers, and team leads.
• Communicate progress, risks, and corrective actions to customer research and program teams.
• Use Python, SQL, or similar tools for quality analysis, defect tracking, and operational reporting.

Requirements

• Direct experience operating production software-engineering data programs for frontier AI models.
• Experience with one or more of the following: coding training or post-training data, coding RLHF or preference data, SWE benchmarks, model evaluations, agentic coding environments, code-generation datasets, or code-repair datasets.
• Proven ownership of contributor sourcing, calibration, task throughput, quality, rework, cost management, and delivery recovery.
• Strong software-engineering judgment and the ability to read code and tests, inspect final artifacts, identify invalid technical work, and make release or rejection decisions.
• Experience managing distributed software-engineering contributors, reviewers, and team leads while maintaining delivery performance.
• Ability to take over an active program quickly, diagnose operational problems independently, and communicate clear decisions and recovery plans.

Preferred Qualifications

• Prior experience from Mercor, Scale AI, Surge AI, Handshake, xAI, or comparable organizations is strongly preferred.

Schedule

• Short-term contract: 3 months.
• Start date: Immediate / next week.
• Minimum commitment: 8 hours per day and at least 40 hours per week.
• Requires 4 hours of overlap with PST.

Evaluation Process

1. Interest check form.
2. AI interview (~20 mins).
3. Delivery interview (~45 mins).