Summary
AgentSimulator inverts the usual business-process-simulation pipeline: instead of discovering a control-flow model and decorating it with parameters, it discovers a multi-agent system directly from an event log, instantiating one agent per resource with its own schedule, activity repertoire, per-agent processing-time distributions, and handover behaviour. Two architectures are discovered and auto-selected per log: orchestrated handovers (global activity-transition probabilities, iterative allocation to the next available agent) and autonomous handovers (agent-conditioned transition probabilities plus pairwise handover probabilities P(aᵢ|aⱼ)). Across 9 public logs (5 synthetic, 4 real, including BPI12W and BPI17W) it most frequently wins on the Chapela-Campa quality metrics — best N-gram distance in 4/9 logs, best cycle-time distribution in 5/9, and the leader on all three temporal distribution metrics — while running 20× faster than Simod and up to 80× faster than DeepSimulator (Production log: 30s vs 10h+ for DSIM on BPI12W). The auto-selection chose orchestrated handovers for 5/9 logs, and switching BPI17W to autonomous halved both CTD (22.75) and RED (26.03). The claim is that resource-first simulation is the honest model for processes where actors, not a workflow engine, decide what happens next.
Key points
- What is extracted from the event log: one agent per distinct resource (plus dummy agents for events with no resource, typically instantaneous or system steps); agent type by clustering resources on activity similarity (Song & van der Aalst); a working schedule per agent; the activity set Alloc each resource was observed performing; and a fitted PDF per (agent, activity) pair over processing times. Distribution family chosen from {exponential, gamma, normal, uniform, log-normal, fixed} by lowest Wasserstein distance.
- Control flow is learned as frequentist next-activity probabilities conditioned on the activity prefix, P(act | σ_prefix), with prefix back-off: unseen prefixes drop their first activity until a seen subsequence is found. No Petri net or BPMN model is discovered at all.
- The autonomous variant conditions transitions on the acting agent, P(act | σ_prefix, a), and adds a handover matrix P(aᵢ|aⱼ) counted as “how often did aⱼ’s activity get followed by aᵢ’s”, divided by aⱼ’s total activities. This is exactly the reassignment/ping-pong structure ITSM logs carry.
- Simulation is a discrete-tick loop: sample case arrivals from a fitted inter-arrival PDF, for each waiting case pick the next activity (globally or from the last active agent), collect eligible agents from Alloc, pick one either by iterative task allocation (ask agents in availability order, or in handover-probability rank order; an agent refuses if the sampled duration collides with a busy or non-working slot) or direct task assignment (sample the successor from the handover distribution and queue the work — the email-forward pattern), then sample duration plus an extraneous-delay PDF.
- Extraneous delays (waiting not explained by contention or unavailability — e.g. waiting on a customer callback) are discovered per activity by the Chapela-Campa & Dumas algorithm, and whether to use them at all is one of only two hyperparameters, tuned by simulating the last 20% of train and comparing cycle time.
- Evaluation is log-to-log distance, not task success: N-gram distance for control flow, absolute/circadian/relative event distributions for time, cycle-time distribution for congestion — a ready-made recipe for scoring “does my simulator replay the real log”. A post-hoc interaction-matrix plot on BPI12W shows AgentSim reproducing the real handover diagonal while Simod produces effectively random interactions.
- Two hyperparameters only (architecture, extraneous delays), selected automatically by holdout on the training log — the model is otherwise parameter-free and interpretable, which the authors argue matters for what-if analysis where deep-learning simulators are useless.
- Limitations the authors name: no multitasking, no batching, no fatigue; handover probabilities ignore the activity (P(aᵢ|aⱼ,act) is left to future work); ACR (432 resources, 954 traces) is their weakest log, hinting the per-agent statistics need volume; no single method wins everywhere.
Relevance
- ITSMBench — the strongest existing answer to “is process mining the bridge from event logs to an agent environment?”, and the answer is partly. It shows concretely what a BPI-style log yields without any external asset: the roster of actors, who can do what, how long each takes, who hands to whom, when they are available, and what the arrival process looks like — i.e. most of the world and the simulated-colleague layer for V2’s multi-agent tracks, learned rather than authored. Its handover matrix is precisely the reassignment structure we want to calibrate against, and its five metrics give a fidelity measure for replay (“logs record what actually happened, so you can replay them and see where things change”) that V2 currently lacks. What it does not cover: there is no environment state, no database, no tools or API surface, no task, no goal, no verifier and no language — its agents are stochastic transition tables, not LLMs, and its “simulation” outputs another event log, not an interactive episode. It also says nothing about whether the discovered process is correct, only that its statistics match.
- WorldSmith — evidence that a meaningful slice of environment construction (actors, roles, capabilities, timing, interaction topology, arrival rates) can be fully automated from data with no LLM in the loop and no human authoring, at ~30s to 9min per log. That is a cheaper and more auditable automation path than agent-authored environments for the parts of the world that are statistical rather than semantic.
- Benchmark Research — the metric suite (NGD / AED / CED / RED / CTD) is a template for scoring realism as distributional distance from a reference log rather than as expert judgement, which is a second, cheaper yardstick alongside practitioner realism.
- Enterprise Environments — home concept since 2026-09-10
Questions it raises for ITSMBench:
- The discovered MAS is a distribution over traces, not an environment. What is the minimum extra layer — entities, tools, state — that turns the AgentSimulator MAS discovered from BPI 2013/2014 into something an LLM agent can act in, and does the transition table then become the colleague simulator’s policy rather than the agent under test’s?
- Orchestrated vs autonomous handovers is auto-selected per log. Which does BPI 2013 VINST pick, and if it picks autonomous, does that empirically justify V2’s decentralised multi-agent track over a single-agent-with-workflow-engine design?
- Fidelity here is log-distance; Sim2Real Gap says binary reward is orthogonal to human-judged realism. If a V2 colleague simulator matched BPI on NGD/CTD, would a practitioner still call it unrealistic, and which of the two yardsticks wins when they disagree?
Notes
(Our take: to add after reading; drafted 2026-09-09, not yet discussed)