Python + English Language Experts - (Freelance, Remote, 20+ openings)

Project Description

This project involves reviewing AI-generated responses, evaluating and refining AI-generated content through Reinforcement Learning from Human Feedback (RLHF). Your primary task will be to assess the accuracy, clarity, and coherence of AI responses, focusing on language use, reasoning quality, and factual correctness.

This role requires a strong command of English, an analytical mindset, and the ability to fact-check across a wide range of topics. You will also engage in light quantitative reasoning (e.g., basic math and logic checks) to ensure responses are both linguistically and logically sound.

Compensation

Compensation is competitive and varies based on education level, institution, and prior experience.

Hiring Profile

• BS, MS, or PhD with significant focus on Computer Science (Required).
• Expert in Python, with a strong grasp of algorithms, data structures, and debugging practices.
• Expert-level English proficiency with excellent writing skills.
• Expert at fact-checking information using public sources across multiple domains.
• Significant experience using large language models (LLMs).
• Strong analytical and structured thinking skills.
• Ability to explain complex programming concepts clearly and accurately.
• Prior experience with RLHF annotation (preferred).
• Prior Software Engineering work experience (strongly preferred).
• Expertise in additional programming languages (nice to have): C#, Java, SQL, C++, TypeScript, PHP, C, Go, Bash, PowerShell, Rust, R.

Application & Onboarding Process

Apply via partner site - Braintrust will send you an email with detailed instructions.

Qualification exam (1–2 hours): evaluates your role-specific expertise. Compensation provided upon passing.

Project exam (1–2 hours): evaluates your adherence to project guidelines and ability to produce Python-specific deliverables. Compensation provided upon passing.

Commitment

We expect a minimum of 17–20 hours per week, however considering the project scope and timelines the preferred and recommended commitment would be 8 hours/day.