JOB-1599
A Series A startup that evaluates and monitors AI agents in production is hiring applied AI engineers. The team wants curious, self-directed builders who can tackle open-ended problems and make its analysis agents both more accurate and more scalable across enormous volumes of data.
Your work
- Improve retrieval and search so the company’s agents can find the right information fast and reach sound judgments
- Create specialized harnesses for inspecting and manipulating agent trajectories, since off-the-shelf tools fall short
- Architect an isolated sandbox in which an evaluation agent operates around the clock, powering self-improving autonomous systems for customers
- Dig into voluminous, untidy production logs of agent interactions, pulling out evaluation signals and lifting data quality via curation, pipelines and benchmarking
- Build agent-based systems that analyze long-running agent behavior in production
Profile
- Experience building AI agents and evals, in applied or generative AI rather than a purely ML-centric or academic path
- Typically one to four years of industry experience, with an upper bound around ten
- Computer science degree; backgrounds from leading universities, top tech companies or high-growth startups are valued
- High agency and comfort with ambiguity
- Bonus: data curation, benchmarking and data pipeline experience; Python
On-site in the San Francisco Bay Area. Visa transfers are straightforward, and new visa sponsorship is possible for exceptional candidates.
