Bench results

Kernel

• date: 2026-09-03
• simulacra: 0.1.0
• bun: 1.3.11
• machine: Apple M5 Max

run	agents	ticks	seconds	events	status	integrity.complete	digest
focal 1k mock	1000	20	6.0	12415	succeeded	true	797393ff8fcc
cohort 100k rule	100000	20	7.4	126	succeeded	true	dfe2a9ccfe96

LLM

• date: 2026-09-03
• simulacra: 0.1.0
• bun: 1.3.11
• machine: Apple M5 Max
• model: deepseek-v4-flash
• endpoint: https://api.deepseek.com/v1

scenario	agents	ticks	llmCalls	promptTokens	completionTokens	cachedTokens	wall s	status	integrity.complete	parseFailures	llmFailures	truncated	rejectedActions	digest
prisoners_dilemma	2	5	5	2016	134	0	4.7	succeeded	true	0	1	0	0	228d09898dd2
echo_chamber	20	3	33	24088	1391	0	9.4	succeeded	true	0	1	0	0	64893fda51d9

total llmCalls: 38 (budget 150)

Comparison

Same machine (Apple M5 Max), same micro-model: 100,000 agents on a random graph with mean degree 8, stance in [-2, 2], stubbornness in [0, 1], 50% activation per tick, opinion update stance += 0.2 * (1 - stubbornness) * (neighborMean - stance), 20 ticks.

system	what it keeps per agent-tick	100k agents, 20 ticks
Mesa 3.5.1 (CPython 3.12)	nothing, in-memory state only	1.38 s
simulacra cohort, in-memory event log	one entry in the tick's observation_batch and decision_batch events	5.8 s
simulacra cohort, SQLite event log (default)	the same 125 events persisted	7.2 s

Reproduce with uv run --with mesa --with networkx --with numpy --with pandas python bench/compare/mesa_echo.py and bun bench/compare/cohort_log_modes.ts.

Prompt footprint per LLM decision, from the recordings in examples/*/recordings against the OASIS README baseline (100 agents, one step, full activation, 335,600 prompt tokens):

system	scenario	prompt tokens per decision
OASIS	Twitter-style feed, README baseline	3,356
simulacra	echo chamber, feed size 5, memory window 2	730
simulacra	prisoner's dilemma	403

Scenario richness differs, so this is a footprint comparison, not a fidelity comparison. Throughput with a real model is bound by the endpoint on every framework: OASIS reports about two days for 100k agents over 10 steps on five A100s with Llama-3-8B, simulacra's DeepSeek runs above sustain about 3.5 decisions per second at concurrency 4.

Summary

Measured on an Apple M5 Max with Bun 1.3.11, see bench/RESULTS.md for the full tables.

run	agents	ticks	wall	notes
focal, mock provider	1,000	20	5.6 s	12,415 events
cohort, rule provider	100,000	20	7.4 s	126 events, one setColumn effect per tick
prisoner's dilemma, deepseek-v4-flash	2	5	4.7 s	5 calls, 2,016 prompt tokens, 0 parse failures
echo chamber, deepseek-v4-flash	20	3	9.4 s	33 calls, 24,088 prompt tokens, 0 parse failures

bun bench/kernel.ts reproduces the first two rows offline. SIMULACRA_LLM_API_KEY=... bun bench/llm.ts reproduces the last two and refreshes the recordings, capped at 150 calls.

Against other frameworks, on the same machine and the same 100,000-agent opinion model over 20 ticks: Mesa 3.5.1 finishes in 1.4 s keeping nothing per agent-tick; the simulacra cohort takes 5.8 s while materializing one observation batch and one decision batch event per tick, and 7.2 s when those 125 events are persisted to SQLite. Per LLM decision, the echo chamber prompt is about 730 tokens against roughly 3,400 in the OASIS README baseline; scenario richness differs, so read that as footprint, not fidelity. Details and reproduction commands are in bench/RESULTS.md.

Reading the numbers

• seconds and wall s are wall-clock time for the whole run, including writing every event to events.sqlite. The comparison table shows what persistence costs: the same cohort run takes 5.8 s with an in-memory log and 7.2 s with SQLite.
• events counts rows in the event log. Focal agents write one observation and one decision per activation; the cohort writes one observation_batch and one decision_batch per tick, which is why 100,000 agents produce 126 events.
• integrity.complete is true when every activated agent ended in a decision or a recorded failure and no tick was cut short. The LLM rows each count one gateway failure under llmFailures and still complete: the failure is an event and a counter, not a hidden retry.
• digest is the sha256 of the event log, the value simulacra digest prints. Two runs of the same scenario and seed must agree on it, so the kernel rows are reproducible to the byte.
• promptTokens per decision is a footprint, not a fidelity measure; the scenarios differ in what each prompt carries.

The commands under each table reproduce it on your own machine: bun bench/kernel.ts for the kernel rows, bun bench/llm.ts with SIMULACRA_LLM_API_KEY set for the LLM rows, and the scripts in bench/compare/ for the comparison.