Summary
A DeepMind position paper arguing that open-endedness — a system continually producing artifacts that are both novel and learnable to an observer — is a necessary property of any artificial superhuman intelligence, and that foundation models finally make open-ended systems practical. It gives a formal, observer-relative definition: novelty means artifacts become less predictable over time even as the observer learns, and learnability means the observer’s predictions still improve with more history. Foundation models supply human notions of interestingness and act as general mutation operators, while open-ended search supplies the discovery that static datasets cannot. The paper sketches research directions (RL, evolutionary search, self-improvement, task generation) and argues open-endedness is also a safety-critical area because humans must remain able to learn from what the system produces.
Key points
- Definition: from an observer’s perspective, a system is open-ended iff the sequence of artifacts it produces is both novel (increasingly unpredictable) and learnable (predictability improves with more history). Novelty or learnability alone is not enough; it is observer-dependent.
- Training on ever larger static datasets will not yield open-endedness; the data is running out and the system has to generate and select its own experience.
- Foundation models and open-endedness are orthogonal and complementary: FMs guide the search toward human-relevant artifacts and generate variations; open-ended algorithms let FMs discover new knowledge.
- Existing open-ended systems (AlphaGo, XLand/AdA, Voyager, FunSearch, Promptbreeder) are narrow; present FMs are not open-ended.
- Paths: RL with self-generated tasks, evolutionary algorithms with FM mutators, self-improvement loops, and the “learning what data to learn from” thread from UED.
- Safety: generally capable open-ended systems raise new risks; explainability matters so the human observer stays in the learnable regime.
Relevance
- Open-Endedness — the anchor paper: it supplies the working definition of open-endedness the concept page should use, connects UED and autocurricula to foundation models, and matches the ICLR 2025 talk notes already on the page.
- World Models — FMs as generators of environments and experience is the bridge between this paper and world-model research.
Notes
(batch ingest; no notes yet)