Paper club

A paper a week on RL environments, benchmarks, simulators and the things that make them hold up. We read it, someone leads, everyone argues. Sessions are recorded and the notes land here afterwards.

It is open. You do not need to work on this full time, and you do not need to have finished the paper.

Next sessions

Each runs for an hour.

• Thursday 5 November, 8:00 AM PT / 4:00 PM GMT / 9:30 PM IST — Building data recipes for terminal agents. Hamish Ivison presents TMax, read against Terminal-World and T1: where tasks come from, how they are checked, what the reward sees, and SFT versus RL. Sign up.
• Thursday 19 November, 9:30 AM PT / 5:30 PM GMT / 11:00 PM IST — User models and user simulators. Persimmon and Turing-RL on training simulated users to be indistinguishable from real ones, at 550B and at 8B. Sign up.

How it runs

• Format. An hour, usually Thursdays at 9:30 AM PT. A short intro on the paper, the rest on whether it holds and what it means for anyone building environments.
• Where. A video call; sign up on Luma beforehand and the link is emailed to you.
• Recording. Sessions are recorded and embedded on the session page. Say so and we keep you out of the recording.
• Notes. Each session gets a page here linking the paper’s page in the wiki.

Reading so far

• 16 October — Sandbox infrastructure for environments at scale. DSec, the sandbox platform behind DeepSeek’s RL: what you need underneath before environment creation can scale.
• 24 September — Autoscaled environments and the Sim2Real gap. DeepSeek-V4.1-Flash on autoscaled task and environment synthesis, paired with Mind the Sim2Real Gap on whether simulated users behave like real ones.
• 17 June — Generating effective learning environments. AI-GAs, Agent Psychometrics and Universal Verifier: what makes a good learning environment, and how to tell training is working.
• 9 June — Multi-turn tool-use benchmarks. VitaBench and WildToolBench, newer multi-turn benchmarks seeded from real user data as Tau2-Bench nears saturation.
• 19 May — Verifiers and trustworthy evals. SWE-Bench Verified, OSWorld-Verified, ComputerRL and BenchGuard on why agent benchmarks mis-score correct agents and how to repair the reward signal.
• 5 May — Gym-Anything: autoscaling CUA. Gym-Anything and its CUA-World: 10K+ autoscaled computer-use tasks across 200 applications, and why scaling needs rigorous QA.
• 22 April — Failure-driven task mining. TRACE on mining tasks from the capabilities a model actually lacks, and RIFT on how LLM-judge rubrics fail and how to catch it.
• 7 April — Tool-graph environment synthesis. Tau-Bench, Tau2-Bench and EnterpriseOps-Gym against AutoForge and ScaleEnv: hand-built enterprise tool-use environments versus fully synthetic tool-graph pipelines.
• 17 March — Priorities in LLM-generated environments. AgentScaler, AutoEnv and OMNI on generating environments, tasks and reward logic with LLMs, and choosing which generated tasks are worth training on.
• 5 March — World models: high-fidelity enterprise environments. Tau2-Bench and EnterpriseBench Corecraft are bottlenecked on human data operations, and LLM-generated environments like Agent World Model and AutoEnv are still simpler.
• 27 February — World models for agents. APEX-Agents, EnterpriseBench Corecraft and Agent World Model: human-built enterprise worlds against LLM-built ones, and what it takes to automate environment and task creation.
• 13 February — SAGE: hierarchical exploration for web agents. SAGE on web agents that train on a self-generated easy-to-hard curriculum, and whether LLM judges can verify the result.
• 11 February — OS-Genesis: reverse task synthesis. OS-Genesis on starting from what the agent did and working backwards to the task.

Suggesting a paper

Bring it to the Discord. We are most interested in work on environment generation, verifier soundness, user simulators, and benchmark validity. Recent things on our list are in the ITSMBench and WorldSmith reading lists.