simulacra

English

中文

English

中文

Appearance

simulacraSocial simulation you can audit

A typed, event-sourced kernel for LLM-driven social simulation. Every observation, decision and state change is a typed event, and the harness asks the question most simulation papers skip. Does the conclusion survive perturbation?

Get started
GitHub
simulacra

Typed world state

Columnar tables are the single source of truth. Agents, modules, and providers read views and emit effects; one resolver applies them.

Event sourcing

Every observation, decision, LLM call, effect, and failure is an append-only event carrying its seed path. Replay is a fold. Same scenario, same seed, same digest.

Recorded LLM calls

Real model outputs are recorded and replayed byte for byte, so audits re-analyze without re-spending tokens.

Failure is data

Parse failures, rejected actions, truncated outputs, budget exhaustion, and circuit breaks are events with counts in every result, never silent fallbacks.

Batch decision providers

Mix LLM, rule, mock, surrogate, archetype, and cache providers in one population; route through TopoSim-style cells or APS-style adaptive prototypes; every provider's own audit numbers land in the metrics.

Audit harness

TRAILS-style perturbation axes across design and representation, replications with seed lineage, Mann–Whitney U, Holm correction, Cohen's d, bootstrap intervals, total variation distance, Wasserstein-1, Cliff's delta, and an evidence grade for the claim.

Three commands ​

Run the echo-chamber example with the deterministic mock provider:

bash
bun run simulacra run examples/echo_chamber/scenario.yaml --seed 7 --provider mock --out runs/echo

Audit the prisoner's dilemma example across persona format, framing, and memory representation:

bash
bun run simulacra audit examples/prisoners_dilemma/audit.yaml --replications 5 --provider mock --out audits/pd

Open the GUI over the runs and audits you just produced:

bash
bun run simulacra serve --data .

With a real model, set SIMULACRA_LLM_API_KEY and drop --provider mock. Any OpenAI-compatible endpoint works. DeepSeek is the default, with thinking disabled in the preset so structured answers fit small token budgets; mlx-lm and LM Studio presets are one flag away. Run bun run simulacra doctor --llm first to check structured-output support, concurrency, and cached-token reporting.

The prisoner's dilemma and echo chamber ship with recordings made against deepseek-v4-flash at the benchmark's size, so replaying them offline means matching that size: --llm-mode replay --ticks 5 for the prisoner's dilemma, and --ticks 3 with population.n 20 for the echo chamber (bun bench/llm.ts runs both). A prompt the recordings do not cover is a replay miss, reported as a failure rather than a silent live call. Use --llm-mode record to make your own at any size.

In numbers ​

401tests pass under bun test
12,415events from 1,000 focal agents over 20 ticks, 5.6 s
100,000cohort agents over 20 ticks, 7.4 s
3 + 5example scenarios and programmatic examples

Timings come from the benchmarks, measured on an Apple M5 Max with Bun 1.3.11.

In the browser ​

The run view: tick timeline with failure counts, the network coloured by stance, one agent's causal chain from observation to effects, metric series, integrity and cost.

Run view

The audit view: plan and evidence grade, perturbation axes, conditions, pairwise tests with Holm-corrected p values, sensitivity ranking and direction consistency.

Audit view

Where next ​

  • Getting started installs the project and explains the three commands.
  • Scenarios and Audits document every field of the two YAML documents.
  • Live report shows an audit report generated when this site was built.