Pod lead - Math (Terminal Bench Science)

Overview

ExpertWoka is seeking an experienced Mathematics expert to lead a small pod of trainers developing realistic, terminal-based scientific and technical tasks for a math-focused benchmark project.

What You'll Do

• Review trainer-created tasks end to end, including problem statements, datasets, environments, reference solutions, and automated tests.
• Verify mathematical correctness, well-posedness, convergence, stability, conditioning, discretization error, and edge-case handling.
• Ensure tasks require genuine multi-step reasoning rather than lookup or a single library call.
• Validate that exact, symbolic, arbitrary-precision, or floating-point computation is used appropriately.
• Check that tolerances are suitable for the problem’s numerical conditioning.
• Design and review graders based on mathematical properties such as residuals, feasibility, optimality certificates, invariants, error bounds, or proof checking.
• Ensure tasks are reproducible through pinned tools, libraries, solver versions, and dependencies.
• Identify and prevent hard-coded solutions, test-file leakage, and other bypass methods.
• Provide precise feedback and track trainer revisions through closure.
• Mentor and onboard trainers on mathematical rigor, task design, coding practices, and evaluation standards.
• Allocate work, monitor quality and throughput, and escalate blockers.
• Maintain pod documentation and collaborate with technical stakeholders to improve benchmark quality.

Requirements

• Ph.D., postdoctoral experience, or equivalent advanced experience in mathematics, applied mathematics, statistics, or a related field.
• Strong programming skills in Python, Julia, C/C++, MATLAB/Octave, or a computer algebra system.
• Strong Linux and terminal proficiency.
• Hands-on experience with numerical methods, scientific computing, symbolic computation, optimization, computational statistics, or mathematical modeling.
• Ability to rigorously evaluate mathematical derivations, algorithms, proofs, and computational results.
• Experience reviewing research outputs, code, mathematical work, or scientific workflows.
• Experience mentoring or leading small technical or research teams.
• Strong written communication and the ability to provide precise, constructive feedback.
• Ability to assess reproducibility, test reliability, and the scientific validity of computational tasks.

Project Details

• Contract length: 3 weeks.
• Start date: immediate / expected next week.
• Compensation: up to $20-$28/hr, final decision based on experience and feedback.
• Availability: at least 4 hours per day, minimum 40 hours per week, with 4 hours overlap with PST.

Evaluation Process

• Interest Check Form (ICF)
• AI Interview
• Delivery Review