Summary
POET (Uber AI) is an algorithm that generates its own problems while solving them: it maintains a population of environment–agent pairs, mutates environments to create new challenges that are neither too easy nor too hard for current agents, optimizes each agent (via evolution strategies) on its paired environment, and periodically tries transferring agents between environments. Tested on a 2-D bipedal-walker obstacle course with tunable gaps, stumps and roughness, POET produces diverse, sophisticated gaits and solves environments that neither direct optimization nor a hand-built direct-path curriculum can solve. The key finding is that transfer between environments — “stepping stones” discovered on unrelated paths — is essential, and which stepping stones matter is unpredictable in advance.
Key points
- Three loops: (1) generate new environments by mutating existing ones, keeping only those that pass a minimal criterion (not trivially easy, not impossible); (2) optimize each paired agent with ES; (3) attempt transfers of agents across environments, keeping a transfer if it beats the incumbent.
- Environments and solutions coevolve in parallel branches rather than a single curriculum; population is capped, with older/easier environments retired.
- Environments POET solves are unreachable by direct optimization from scratch, and often unreachable even by a curriculum that walks the direct path from easy to the target — because the useful stepping stones lie off that path.
- Diversity of behaviors (walking, hopping over stumps, clearing gaps) emerges in a single run.
- Inspirations: minimal criterion coevolution, novelty search with local competition, MAP-Elites, Innovation Engines.
- Limitations the authors state: the environment encoding is bounded (max gap/stump size) so difficulty maxes out; body morphology is fixed. Richer, unbounded encodings (e.g., CPPNs) are the proposed next step.
Relevance
- Open-Endedness — the classic open-ended algorithm from the artificial-life side of the field; where PAIRED/PLR use regret, POET uses a minimal criterion plus transfer, and it is the clearest demonstration that stepping stones off the direct path are what make hard problems reachable.
- Unsupervised Environment Design — home concept since 2026-09-10
Notes
(batch ingest; no notes yet)