JOB-1496
Our client is a venture-backed data infrastructure company serving the AI industry. It is starting a dedicated line of work around model benchmarks and evaluations and wants its first machine learning engineer for that effort, working hand in hand with the line’s general manager and in-house researchers.
The work
- Work with the business lead and first customers to scope and create benchmarks and evals for varied domains and data types
- Own the backend base layer covering data pipelines, runtime environments, storage and job orchestration
- Set up isolated sandboxes for agent-style evals that call tools, run code or span many steps
- Carry the engineering side of each customer engagement from start to finish
- Spot recurring evaluation patterns and missing infrastructure that suggest product ideas
Profile
- Four to eight years working as an engineer, with hands-on evaluation of ML models
- Benchmarks and evals designed from scratch, not merely existing frameworks run as-is
- Past ownership of backend systems and infrastructure, for instance test harnesses, parallel runs, checkpointing and cost tracking
- Experience building human-data pipelines or evals for LLMs, and shipping to tight customer deadlines
- Customer-facing engineering experience, excellent writing, and comfort with ambiguity
- Python background
Remote within the United States. Visa sponsorship and transfers are not available.
