Senior ML Inference Engineer (Real-Time AI Startup)

Our client is a venture-backed AI company whose products depend on fast, reliable multimodal model serving. It is hiring senior engineers to make model inference quicker, cheaper and dependable at large scale, and it cares more about what candidates have actually built than about a conventional resume.

What the job involves

• Optimize how models are served, using modern frameworks such as vLLM or TRT-LLM
• Apply acceleration methods including speculative decoding, paged attention, continuous batching, quantization, distillation and caching
• Write high-performance code, profile it carefully and push NVIDIA GPUs to their limits
• Scale serving across multiple GPUs and nodes with tools like Kubernetes and Ray, while holding up under thousands of simultaneous connections
• Move a model out of research, package it in containers, tune its serving and keep it reliable in production

Background we value

• Five or more years as an ML inference or systems engineer at senior level
• Hands-on inference optimization experience and strong systems skills in CUDA, C++, Rust or carefully optimized Python
• Distributed systems experience, plus public work such as open-source contributions or technical write-ups
• A PhD in a quantitative field, or comparable hands-on time building backend and ML systems
• Comfort with ambiguity, a focus on shipping stable results, and a habit of questioning architecture when a better path exists

Remote within the United States. Visa transfers considered.