# Make Experiments Reproducible
# Book: Cellular Automata From First Principles
# Chapter URL: https://programmer.ie/books/cellular-automata-from-first-principles/55-chapter/
# Prose fingerprint (sha256, normalized): fb445abedeca9eb4ef305e71e0a248c2759b1342d21be7d1b216d19cc338df3c
# Snapshot generated at build time by Hugo. This is the complete rendered prose
# of the chapter; it is not a transcript of the source file.

By this stage the book contains many systems whose behavior depends on parameters, seeds and implementation choices.
If we cannot reconstruct a run, we cannot really compare it.
Treat configuration as data from dataclasses import dataclass, asdict @dataclass(frozen=True) class ExperimentConfig: model: str width: int height: int steps: int seed: int backend: str dtype: str params: dict = None Then extend with model-specific parameters rather than hiding them in notebook cells.
Seed every relevant source of randomness import random import numpy as np import torch def seed_everything(seed): random.seed(seed) np.random.seed(seed) torch.manual_seed(seed) if torch.cuda.is_available(): torch.cuda.manual_seed_all(seed) Determinism can still depend on backend and operation, but explicit seeding is the minimum contract. The documented limits are worth stating plainly: identical seeds do not guarantee identical results across PyTorch releases, platforms, or CPU-versus-GPU execution — and some CUDA operations (convolution benchmarking, nondeterministic backward passes such as the one behind padded convolutions) need explicit flags beyond seeding. Seeding is necessary, not sufficient.
Save the exact configuration import json from pathlib import Path def save_config(config, path): Path(path).write_text( json.dumps(asdict(config), indent=2), encoding="utf-8", ) A figure without the parameters that generated it is a dead artifact.
Record software context Useful experiment metadata includes:
Git commit Python version library versions device operating system random seed input data hash This does not guarantee identical results forever.
It makes discrepancies explainable.
Give each run an identity A simple run directory might look like:
runs/ 2026-08-10T205700_rule110_seed42/ config.json metrics.json final.npy frames/ benchmark.json Better still, derive a stable ID from normalized configuration plus code version. The full reproducibility chain, with each link’s failure mode:
flowchart LR C[config + seed + version] --> R[run] R --> A[artifacts: raw + metrics] A --> P[replay from config] P --> K{bit-identical?} K -->|yes| T[trustworthy result] K -->|no| D[debug: backend, version, or nondeterminism] Input Why needed What can still vary deterministic seed selects one realization nothing, if backend is deterministic backend + device execution semantics differ CPU vs GPU numerics library versions algorithms change across releases cuDNN benchmarking choices saved config reconstructs the run code changes since (pin the commit) raw artifacts re-derive figures without rerun storage cost, format rot Separate inputs, outputs and derived artifacts inputs → seeds, maps, targets outputs → raw states, checkpoints, metrics derived → plots, animations, summary tables That distinction matters because a graph should be regenerable from raw results.
Replay should be a first-class operation Replay is not an aspiration here — it is a function, shown complete for a Life configuration and verified bit-identical across repeated runs:
def replay(config): seed_everything(config.seed) rng = np.random.default_rng(config.seed) state = (rng.random((config.height, config.width)) < 0.4).astype( np.uint8 ) for _ in range(config.steps): state = life_step_np(state) return state (life_step_np is Chapter 51’s NumPy life_step; the name marks it as the NumPy version, since Chapter 52 defines a tensor one.)
The strongest test is simple:
Can another process reconstruct this run from the saved configuration? (Verified: two replays of a 48×48, 100-step configuration produce byte-identical final states, and the config serializes to JSON — reconstructability demonstrated, not asserted.)
Reproducibility enables debugging When an interesting organism disappears after a code change, compare:
same config old commit new commit Now the change in behavior is evidence about the implementation.
Without replay, it is merely a memory that something once looked different.
Reproducibility enables search Search produces thousands of candidates.
A candidate record should contain enough information to recreate it:
candidate = { "score": 0.812, "seed": 3917, "parameters": params, "steps": 800, "model": "lenia", } Never save only the top PNG.
The experiment is the unit of knowledge A useful mental model is:
claim ↓ configuration ↓ execution ↓ raw result ↓ metric / figure That chain is what lets the book move from demonstrations toward real computational experiments.
Next we will use this structure to run parameter sweeps and benchmarks systematically.
Research The Turing Way — Guide for Reproducible Research. The community handbook this chapter’s practices implement: configuration capture, run identity, inputs/outputs/derived separation, and the claim→configuration→execution→result chain as the unit of knowledge. Read it before inventing a lab workflow from scratch. https://book.the-turing-way.org/reproducible-research/reproducible-research/
PyTorch documentation: Reproducibility notes. The exact limits of seed_everything-style discipline: what seeding controls, where cuDNN benchmarking and nondeterministic algorithms break it, which flags restore determinism, and why CPU/GPU parity is not promised. Required reading before claiming any torch result reproduces. https://docs.pytorch.org/docs/stable/notes/randomness.html