Data Annotation Generalist Specialist

1 day ago Be among the first 25 applicants

This range is provided by Mercor. Your actual pay will be based on your skills and experience — talk with your recruiter to learn more.

Base pay range

$25.00/hr - $35.00/hr

About The Job

Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey .

Position: Preference Ranking Model Evaluator

Type: Freelance

Compensation: $25–$35/hour

Location: Remote

Commitment: 10–20 hours/week

Role Responsibilities

• Evaluate and compare AI-generated responses based on quality, coherence, and helpfulness.
• Assign preference rankings to pairs or sets of model outputs.
• Follow detailed labeling guidelines and adjust based on evolving criteria.
• Provide brief written explanations for ranking decisions when required.
• Flag edge cases or inconsistencies in task design or model output.
• Work independently and asynchronously to meet deadlines while improving AI model performance.

Must-Have Qualifications

• Prior experience in data labeling, content moderation, or preference ranking tasks.
• Excellent critical thinking and reading comprehension skills.
• Comfort working with evolving guidelines and ambiguity.
• Strong attention to detail and consistency across repetitive tasks.
• Availability for regular part-time work on a weekly basis.

Compensation & Legal

• Payments issued weekly via Stripe Connect.
• Classified as an independent contractor.

Application Process (Takes 20–30 mins to complete)

• Submit your resume.
• Complete a short form to highlight your relevant experience.
• You may be asked to complete a brief assessment to evaluate task fit.

Resources & Support

• For details about the interview process and platform information, please check:
• For any help or support, reach out to:

PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.