Marcus Thorne

I run the stack from training jobs to inference: GPU pools, model registry, cost attribution, and safe rollout for dozens of internal models.

• Cut inference spend 35% with batching + speculative decoding patterns
• Unified ONNX + vLLM serving behind a single gateway
• Built drift monitors tying embeddings quality to business KPIs

1. Head of ML Infrastructure

   Harborstack · 2021 — Present

   Head of ML infrastructure for a regulated ML platform vendor: ONNX + vLLM behind a unified gateway, per-tenant isolation, and cost attribution by model version and team. Cut inference spend ~35% via batching, KV-cache reuse, and speculative decoding on eligible workloads. Partnered with security for key management and quarterly pen tests. Built drift monitors linking embedding quality shifts to business KPI alerts; integrated with PagerDuty and exec dashboards.