LLM Evaluation Engineer

Job description

Scale AI is hiring a LLM Evaluation Engineer (Full-time) hybrid. This llm engineer role is part of the AI hiring market in San Francisco (California). Design benchmarks and human-in-the-loop eval pipelines for frontier model customers. Listed compensation: $150k – $205k / year. Stack: Evals, Python, LLM, RLHF.

LLM Engineer profiles are in high demand in San Francisco: San Francisco and the Bay Area host the largest share of US AI hiring: LLM, MLOps, foundational research, and GenAI products. OpenAI partners, Databricks, Scale AI, and hundreds of…

The team

Scale AI — AI team

Responsibilities

• Ship production AI features with measurable quality and latency targets
• Partner with product and design on roadmap and customer feedback
• Improve observability, evals, and safe rollout practices

Requirements

• Strong Python or TypeScript fundamentals and 3+ years relevant experience
• Hands-on LLM, ML, or data platform work in production
• Clear written communication and async collaboration

Tech stack

Benefits

Hiring process

1. 1Recruiter screen (30 min)
2. 2Technical interview (60 min)
3. 3Team fit
4. 4Offer

Experience: Mid · 3+ years · Education: BS/MS CS, ML, or equivalent experience

Salary guide — LLM Engineer in San Francisco →

FAQ — LLM Evaluation Engineer

What is the salary for LLM Evaluation Engineer at Scale AI?

Listed range: $150k – $205k / year (Full-time). Packages often include PTO, remote options, and equity depending on the company.

Where is the LLM Evaluation Engineer role based?

San Francisco, CA, San Francisco (California). Work mode: Hybrid.

How do I apply to Scale AI?

Create a free Ganloss candidate account, then apply in one click. Job reference: EXP-2026-0012. Process: Recruiter screen (30 min) → Technical interview (60 min) → Team fit → Offer.

Is Scale AI hiring other AI roles?

Scale AI (AI data & eval) hires across LLM, ML, data, and AI product. See all openings on the Ganloss company page.

Your profile is sent directly to Scale AI's hiring team.