Cantab Reasoning Institute is hiring a Senior Research Engineer — LLM evaluation & benchmarks on the R&D team (Hybrid · Cambridge, United Kingdom). This is a hands-on role in LLM & generative AI—not a generic “AI enthusiast” post.
You will own LLM features, retrieval, agents, and evaluation end to end: problem framing, offline evaluation, safe rollout, and iteration from production signals. Success is measured with clear metrics (quality, latency, cost, or business KPIs)—not slide decks alone.
Cantab Reasoning Institute builds internal platforms for LLM evaluation, safety testing, and benchmark governance. We sit between research, product, and policy teams to keep releases evidence-based.
Hiring a Research Engineer in Cambridge (hybrid). You will turn evaluation ideas into reliable systems—datasets, harnesses, dashboards, and CI integrations.
What you will do
- Design golden-set and adversarial suites with clear coverage maps across capabilities and risk categories.
- Implement LLM-as-judge flows with calibration against human raters; track drift over model versions.
- Integrate evaluation gates into release trains; surface actionable diffs to PMs and researchers.
- Collaborate with red teams to reproduce issues and convert findings into regression tests.
- Publish internal playbooks and lightweight tooling for partner teams.
What we look for
- Strong Python engineering and experiment discipline; comfortable reading research papers and reproducing methods.
- Experience with prompt/model iteration cycles and structured logging for ML runs.
- Clear writing; ability to explain statistical limitations to leadership.
- 4+ years in ML, NLP, or evaluation-heavy roles.
- UK working rights.
Nice to have
- Background in HCI or annotation pipeline design.
- Experience with multilingual evaluation or cultural bias auditing.