About this opportunity
Overview
Rex is seeking experienced software engineers to help train and evaluate AI models using real-world programming work. The role centers on creating coding tasks, checking AI-generated code, and producing the reference solutions and tests that are used to measure model performance.
What You'll Do
- Design realistic programming tasks such as bug fixes, feature work, refactoring, and performance improvements with clear specifications.
- Set up reproducible development environments using Git and Docker, and write automated test suites to validate solutions.
- Create high-quality reference solutions and clearly explain the design choices behind them.
- Review AI-generated code for correctness, performance, security, and readability.
- Compare model outputs, rank them, and provide clear written explanations for your judgments.
- Identify recurring failure patterns in model responses and share them with the project team.
- Participate in calibration and review sessions and follow project guidelines to keep evaluations consistent.
Requirements
- A degree in computer science or a related field, or equivalent professional experience.
- At least 3 years of professional software engineering experience.
- Strong Python skills, plus proficiency in at least one of Java, C++, TypeScript, Go, or Rust.
- Experience using Git, Docker, and automated testing.
- Ability to read and navigate unfamiliar codebases.
- Fluent written English and clear technical communication.
- Ability to work independently, meet deadlines, and stay reliable and detail-oriented in a remote setting.
Preferred Qualifications
- Open-source contributions or code review experience.
- Experience with test automation, CI/CD, or developer tooling.
- Prior LLM evaluation or data annotation experience.
- Experience with large or distributed codebases.
Legal and Compliance
All submitted work must be lawful, and you must have the right to share it. You must not submit confidential or proprietary material from an employer, client, or third party, or personal data without a lawful basis. Rex may verify identity, credentials, ownership, and licensing, and may reject or remove submissions that do not comply. Engagement and payment are subject to verification and to applicable sanctions, export-control, tax, and employment laws. The contractor agreement will define ownership and licence terms for the work delivered.
