Pacific Context AI is hiring a Staff Staff Software Engineer — LLM platform & APIs on the Core platform team (Hybrid · San Francisco, United States). This is a hands-on role in LLM & generative AI—not a generic “AI enthusiast” post.
You will own LLM features, retrieval, agents, and evaluation end to end: problem framing, offline evaluation, safe rollout, and iteration from production signals. Success is measured with clear metrics (quality, latency, cost, or business KPIs)—not slide decks alone.
Pacific Context AI provides APIs and SDKs for enterprises launching copilots on private data. Reliability, predictable cost, and safe defaults matter as much as model quality.
We are hiring a Staff Software Engineer in San Francisco (hybrid) to evolve our core LLM gateway, streaming layer, and multi-tenant controls.
What you will do
- Design high-throughput request paths with caching, adaptive batching, token accounting, and backpressure strategies.
- Implement streaming (SSE/WebSocket) clients and server contracts with idempotency keys for long jobs.
- Partner with security for tenant isolation, key management, and abuse detection.
- Mentor engineers on performance profiling and production readiness checklists.
- Shape technical standards for new model integrations and evaluation hooks.
What we look for
- 8+ years building distributed systems; strong Python and one systems language (Go/Rust) preferred.
- Experience operating APIs with strict SLOs and on-call culture.
- Working knowledge of LLM provider APIs, rate limits, and fallback routing.
- US work authorisation; willingness to be in SF several days per week.
Nice to have
- Prior startup experience from seed to Series C scale.
- Contributions to gateway or API gateway open-source projects.