MSI TalentHub
← Todas las vacantes abiertas

Vacante abierta

Senior Software Engineer - AI Evaluation / Coding Agents

Turing

Remote · San Francisco5+ years$100-$150/hr

Senior Software Engineer - AI Evaluation / Coding Agents

Professional Summary

Turing is one of the world's leading AGI infrastructure companies, working with frontier AI labs to accelerate model development through high-quality training data, evaluations, and engineering talent. We are seeking experienced, hands-on Senior Software Engineers to evaluate and improve AI coding models and autonomous coding agents. Rather than building standard production applications, you will work directly with coding agents across real-world repositories to evaluate their outputs, determine technical correctness, identify failure modes, and build evaluation signals that advance AI model performance.

Key Responsibilities

  • Design, refine, and iterate on rubrics and evaluation criteria for preference data and coding agent assessments.
  • Review and label code data with a high quality bar, identifying subtle edge cases, architectural oversights, and technical defects.
  • Build and maintain data generation, collection, and evaluation infrastructure and pipelines.
  • Synthesize qualitative data insights into structured write-ups, engineering updates, and technical recommendations.
  • Collaborate closely with AI researchers and engineers to convert technical judgment into scalable model evaluation workflows.

Technical Requirements (Must-Have)

  • 5+ years of hands-on software engineering experience in production environments.
  • Programming Languages: Strong proficiency in Python, TypeScript/JavaScript, Go, or another major production programming language.
  • Codebase Experience: Demonstrated experience navigating and contributing to substantial, complex real-world codebases.
  • Code Quality & Judgment: Exceptional code-review skills and technical judgment.
  • Communication: Ability to clearly articulate and justify why code implementations are correct, incorrect, or sub-optimal; excellent written communication skills.

Desired Skills (Nice-to-Have)

  • Experience using modern LLMs or AI coding tools (e.g., Cursor, GitHub Copilot).
  • Experience in LLM evaluation, autonomous coding agents, RLHF, preference data design, rubric creation, or model post-training.

Engagement Details

  • Role Type: Independent Contractor / Freelance (Approx. 3 months)
  • Schedule: 20–40 hours/week (Minimum 6 hours/day Pacific Time overlap)
  • Location: Remote (North America only)
  • Compensation: $100 – $150 / hour