Repository Software Engineering Expert

Help us find where advanced coding agents break in repository software engineering. Build or solve realistic tasks, stress-test generated code, and explain failures with reproducible evidence. Remote project-based work can pay up to $45/hour, depending on the project and demonstrated expertise.

What you'll do:

• Create, solve, or review realistic tasks involving real multi-file repositories, issue diagnosis, code changes, regression tests, and maintainability.
• Test AI-generated code and agent traces for correctness, edge cases, regressions, unsafe assumptions, and incomplete fixes.
• Write strong tests, reference patches, expected outcomes, and concise explanations of the failure mode.
• Review peer-authored tasks for realism, reproducibility, difficulty, and clean evaluation criteria.

What we look for:

• Strong practical depth in real multi-file repositories, issue diagnosis, code changes, regression tests, and maintainability.
• Ability to navigate unfamiliar code or specifications and communicate technical decisions in clear written English.
• Verifiable evidence such as meaningful GitHub work, shipped software, a competition profile, technical writing, or a portfolio. Confidential industry work can be described in the application without exposing proprietary material.
• Prior AI-evaluation experience is useful but not required; strong engineering judgement and testing discipline matter more.