Work Type
Remote
Location
Global
Compensation
Not specified
About this opportunity
Overview
CNTXT AI is hiring a remote hourly contractor to evaluate AI-generated Python code and create test cases that assess coding accuracy and reasoning quality. The role focuses on reviewing code outputs, identifying issues, and helping improve how AI systems perform on Python programming tasks.
What You'll Do
- Review AI-generated Python code for correctness, efficiency, readability, and alignment with real-world engineering standards.
- Identify bugs, logical mistakes, edge cases, and anti-patterns in model outputs.
- Explain corrections clearly in writing.
- Use structured evaluation rubrics to score code quality consistently across different task types.
- Develop prompts and test cases that probe AI accuracy on Python tasks, including multi-step problems based on real codebases.
- Compare AI-generated solutions based on correctness, internal consistency, and prompt adherence.
Requirements
- Background in Computer Science, Software Engineering, or a closely related field preferred.
- Strong proficiency in Python, including data structures, algorithms, standard libraries, performance optimization, and idiomatic coding practices.
- Ability to read, write, and critically evaluate Python code across different levels of complexity.
- Full professional English proficiency.
- Exceptional attention to detail.
- Reliable, self-directed work style with consistent output quality in a remote asynchronous workflow.
Preferred Experience
- 2+ years of professional software engineering experience with Python.
- Prior experience in AI data training, annotation, or model evaluation.
About CNTXT AI
CNTXT AI develops artificial intelligence products and data solutions with a focus on accuracy, safety, and global relevance. Its work includes data services, custom AI solutions, and proprietary AI products, with expertise in Arabic-native and secure sovereign solutions.
