Back to all jobs
Mercor

AI Developer Trace Task Auditor

Pay

$70–90/hr

Location

USA

Engagement

Contract · Remote

Posted

2 mo ago

About the work

Evaluate the quality and correctness of AI-assisted software-development traces used to train and evaluate a frontier AI lab's models. You'll assess end-to-end coding sessions produced with AI-assisted developer tools — judging correctness, workflow soundness, and reasoning — and provide clear, rubric-based written feedback. Basic Qualifications • 3+ years professional software development • Hands-on experience with AI-assisted coding tools and agentic / spec-driven workflows (Cursor, GitHub Copilot, Claude Code, or similar) • Strong code-reading and debugging skills across full-stack or backend systems • Ability to evaluate multi-step coding trajectories for correctness and best practice Preferred Qualifications • Experience with Kiro or Amazon CodeCatalyst • Prior work evaluating or grading AI-generated code • Contributions to developer tooling

Good fit if you have

  • • Any-level experience in Software & Coding, Evaluation & Rating, Science & Mathematics
  • • Fluency in English
  • • A reliable laptop and internet connection
  • • Comfort writing clear, structured explanations
Software & CodingEvaluation & RatingScience & MathematicsEnglish

Applying is free. Shared links open this job page, and Apply now uses our referral link when available.

Help us measure outcomes

No name or application details are collected.