Software Engineer, Inference Systems (AI Infrastructure Startup)

A small AI infrastructure startup hosts open-source language models for customers and uses automated optimization to squeeze more performance out of varied hardware. It is hiring several systems-minded engineers who like working close to the machine and who are happy to talk with customers when needed.

What you’ll own

• Bring up support for newly released open models as fast as possible, tuned for response time and throughput
• Improve the serving stack, including batching, KV caching, speculative decoding and quantization
• Write and tune GPU kernels in CUDA, HIP and Triton across NVIDIA, AMD and several accelerator families
• Design, deploy and run clusters that mix hardware from different vendors
• Keep production inference reliable, observable and economical across a heterogeneous fleet

Who thrives here

• One to six years of software engineering focused on systems near the hardware
• A real understanding of computer architecture, memory hierarchy and operating system internals, beyond application code
• Backend, infrastructure or systems experience at a strong technology company or early-stage startup
• Evidence of standing out: competition results, a hard system owned end to end, or quick progression
• Daily use of AI coding agents to speed up your own work
• Clear communication, since a slice of the job involves customers
• Python and cloud familiarity; experience with GPU programming or inference optimization is a plus

On-site in the San Francisco Bay Area, five days a week, with a fast start expected. Visa transfers for people already in the US are possible; no overseas sponsorship.