Research Scientist, Safety Evaluations (Applied AI Research Lab)

A young applied AI research lab serving frontier model developers is looking for a hands-on researcher, not a manager, to start a safety evaluation program. The first focus is measuring how well models can carry out and speed up AI research themselves, followed by studying risky model behavior.

The work

• Design benchmarks that gauge how well models can automate research and improve themselves, with explicit tasks, reference baselines, resource limits and definitions of success
• Build test environments that surface exploitation of reward signals, sabotage, deceptive conduct and failure to respect stated limits
• Examine ways results can mislead, such as data contamination, weak graders, ambiguous tasks and shortcuts
• Test today’s models, study their transcripts and failure cases, and follow how abilities evolve from one model to the next
• Partner with engineers to convert research designs into environments others can rerun, with transparent scoring and written caveats
• Publish benchmark releases and reports, and advise lab partners on what the evidence means
• Help steer the safety research agenda; some of the environments may later feed into model training

Profile

• One to ten years as a research scientist or alignment researcher, with recent hands-on safety work on frontier models
• Experience with post-training and reinforcement learning
• Contributions to recognized benchmarks or papers at major ML venues
• Python and PyTorch
• Solid grasp of current frontier capabilities and where they are heading
• Work on AI R&D or self-improvement evaluations, pre-deployment testing, or a PhD is a plus

On-site in the San Francisco Bay Area. Visa transfers considered; no new visa sponsorship.