Computer Science Specialist

About The Job

Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark, General Catalyst, Peter Thiel, Adam D'Angelo, Larry Summers, and Jack Dorsey.

Position: AI Research Evaluator

Type: Contractor
Compensation: $50–$60/hour
Location: Remote
Commitment: 30 hours/week

Role Responsibilities

• Review and audit submitted data, including prompts, solutions, final answers, references, and metadata.
• Evaluate task quality using structured turn-level and task-level rubrics, ensuring clarity, correctness, and logical consistency.
• Verify the accuracy of solutions, reasoning steps, and chain-of-thoughts, identifying flaws, gaps, or inconsistencies.
• Assess coding problems by validating test cases, solution correctness, and language consistency.
• Provide concise, well-reasoned written feedback to justify evaluation decisions.
• Work independently and asynchronously following provided tools and workflows.

Qualifications

Must-Have

• PhD (or current PhD candidacy) or Master's in Mathematics, Computer Science, Applied Mathematics, or a related STEM field.
• Expertise in one or more domains, including Mathematics (algebra, geometry, number theory, combinatorics, probability, graph theory), Coding (data structures, algorithms, application frameworks, testing & quality assurance), General domains (summarization, open/closed Q&A, classification, reasoning tasks).
• Strong analytical skills with the ability to evaluate multi-step reasoning.
• Excellent written communication and attention to detail.
• Ability to work independently in a fully remote environment.

Preferred

• Basic knowledge of Python and testing frameworks (e.g., Pytest, JUnit).

Compensation & Legal

Contractor position via Mercor, paid hourly.
Competitive rates based on expertise: $50–$60/hour.
Weekly payments processed securely through Stripe Connect.

Application Process (Takes 20–30 mins to complete)

• Upload resume
• AI interview based on your resume
• Submit form

Resources & Support

For details about the interview process and platform information, please check: https://talent.docs.mercor.com/welcome/welcome
For any help or support, reach out to: [email protected]

PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.