Software Engineer, RL Environments

Job Description

Job Description

San Francisco, CA · On-site · Full-time
Compensation: $180,000–$220,000 + competitive equity

About the Company

An early-stage (post–Series A) company building the training data and evaluation infrastructure that frontier AI labs use to improve their models — designing high-signal datasets and running rigorous evaluations that go beyond static benchmarks. A small team where individual contributors have direct impact on how the next generation of models learns. The company has raised $30M (~$300M valuation), with a founding team drawn from Jane Street, Citadel, Google, Goldman, and Stanford AI Lab.

Founded 2025 · 11–50 people · Industry: Consumer Tech

The Role

As a SWE (Environments), you'll design the datasets and evaluation rubrics that directly influence how frontier models learn — going from hypothesis to live experiment quickly, with output feeding directly into model training runs at scale.

What you'll be doing

• Design data slices and explore data shapes that expose meaningful model failure modes across domains like finance, code, and enterprise workflows
• Build and refine evaluation rubrics and reward signals for RLHF and RLVR training pipelines
• Model annotator behavior and run experiments to improve different model capabilities
• Develop quantitative frameworks for measuring dataset quality, diversity, and downstream impact on model alignment and capability
• Create and manage both real-world and synthetic data pipelines
• Partner with lab research teams to translate their training objectives into concrete data and evaluation specifications

Tech stack: Not specified

Requirements

• 1–4 years of software engineering experience with strong technical depth
• Design targeted data slices that surface model failure modes across high-stakes domains (finance, code generation, enterprise workflows)
• Build and iterate on evaluation rubrics and reward signals powering RLHF and RLVR training pipelines
• Develop quantitative frameworks to measure dataset quality, diversity, and downstream impact on model alignment and capability
• Own end-to-end real world and synthetic data pipelines, from scoping with research teams to production-ready evaluation specs
• Run annotator modeling experiments to improve model capabilities across task types

Green Flags

• Experience at RL environment companies
• Background in AI safety or benchmarking organizations like METR or Artificial Analysis
• Genuine obsession with how data structure, selection, and quality drive model behavior
• Ability to design lightweight experiments and move fast
• Former founders or early engineers at early stage startups
• Demonstrated ability to work hard, learn fast, and care deeply about details

Red Flags

• Pure research profile with limited engineering output, this is a SWE role, shipping matters
• Looking for standard product engineering work — the real scope is data pipelines, reward modeling, and eval infra

Why Join

• Outsized total cash: base plus substantial profit share, plus competitive equity
• Direct impact on frontier AI model development, working with the world's leading AI labs
• High ownership on a small, early team — scope, build, and ship end to end

Details

• Location: San Francisco, CA
• Work policy: On-site
• Compensation: $180,000–$220,000 + equity
• Visa sponsorship: O-1, OPT
• Employment type: Full-time