The paper
Hamish Ivison, one of the two first authors, presents TMax. We read it alongside two other recipes for training terminal agents, Terminal-World and T1, and take the session one ingredient at a time.
TMax is the open baseline: Tmax-15k, 14,600 synthetic terminal environments, each sampled across nine axes and instantiated by a generator model into a Dockerfile, a unit-test verifier and instructions, plus an outcome-only RL recipe that takes Qwen 3.5 9B to 27% on Terminal-Bench 2.0 TMax. Terminal-World builds environments from human-authored agent skills instead, deriving task, sandbox, verifier and a teacher’s execution guideline from one skill, and trains by SFT alone: 5.7K trajectories take a 32B model to 31.5 on Terminal-Bench 2.0 Terminal-World. T1 is the large-scale RL end: a 122B MoE trained with PPO for 300+ tool-call turns per task on an audited pool of synthesized tasks, lifting its SFT checkpoint from 49.4% to 64.0% on Terminal-Bench 2.1 T1.
| Ingredient | TMax | Terminal-World | T1 |
|---|---|---|---|
| Where tasks come from | Nine sampled axes | Human-authored agent skills | Extending a reference solution |
| Quality control | Image must build; RL drops all-zero groups | Generate–verify–repair; verifiers must fail on the initial state | Reference must pass a held-out verifier; LLM audit rejects hidden requirements, leakage, shortcuts |
| Reward | Binary outcome | None (SFT only) | Count of passing verifier assertions |
| Training | RL only | SFT only | SFT, then PPO |
| Scale and cost | 14.6K tasks; ~$3,150 per run on Daytona | 5,723 environments at ~$0.17 each | 15K audited tasks |
Sources for the table: TMax, Terminal-World, T1.
Questions for Hamish:
- Is RL filtering a substitute for validation? TMax checks only that the image builds and lets all-zero groups fall away TMax. Terminal-World validates every artifact Terminal-World, and T1 rejects tasks whose verifier and instruction disagree T1. After RL, Tmax-9B replaced a test checker with a no-op and built stub binaries on Terminal-Bench 2.0 — runs that scored 0, so they surfaced only at eval TMax. Where would a validation step pay for itself?
- How much does the reward’s granularity matter? T1 ran binary-reward RL on TMax-15k from its base model: 43.8% to 47.2%, never reaching its SFT checkpoint; its gains came from counting passing assertions on its own data T1. Is that the data, the reward or the scale?
- Are verifiers too strict or too loose? LLM judges rated 67.7% of the trajectories Terminal-World’s pytest verifiers failed as task-completing Terminal-World, while TMax’s hacking cases are verifiers passing what they should not TMax. Which failure costs a recipe more?
- SFT or RL? SFT warm starts hurt Qwen 3.5 9B in TMax TMax, Terminal-World reports no RL at all Terminal-World, and T1 depends on an SFT checkpoint before PPO T1.
- Can a synthetic recipe beat its generator? The TMax authors leave it open TMax; it is the question we kept returning to on Autoscaled Environments and the Sim2Real Gap.
Discussion
(none)
What we took from it
(none)