Find the loops nobody wrote down

Some work is not finished and nobody wrote that down. The chapter’s claim is that a memory can infer those open loops, and that the interesting measurement is not how many it finds but how many it invents. The suite here reports both, and they are not the same number.

Run it

git clone https://github.com/ernanhughes/project-memory.git
cd project-memory
git checkout 0a6419ab0d6f65e491374b971d214255f2b0d8e8
python solution/run_ch9.py --outdir ./runs/ch9-demo

Python 3.11 or later. The suite is fixture-level and deterministic: zero model calls, no database, no service. It writes a new run directory.

Expected outcome

The system finds every open loop in the fixture and closes none of them wrongly:

{
  "status_accuracy": 1.0,
  "cross_artifact_closure_accuracy": 1.0,
  "side_effect_completion_accuracy": 1.0,
  "open_loop_precision": 0.8,
  "open_loop_recall": 0.8,
  "false_open_rate": 1.0,
  "canonical_status_accuracy": 1.0,
  "false_closed_rate": 0.0,
  "stale_open_rate": 1.0,
  "reverification_repair_rate": 1.0,

The two numbers to read together are open_loop_recall at 0.8 and false_open_rate at 1.0. Recall says most of the real loops were found. The false-open rate says every loop it reported that was not real was reported as open. A system tuned only on recall would look excellent here and be unsupportable in use.

The rest of the suite is the part that keeps the claim honest:

  "bitemporal_status_accuracy": 1.0,
  "historical_status_accuracy": 1.0,
  "history_gap_calibration": 1.0,
  "search_footprint_coverage": 1.0,
  "projection_rebuild_equivalence": 1.0,
  "maintenance_derivation_agreement": 1.0,
  "historian_colleague_demo": 1.0,
  "deadline_flag_accuracy": 1.0,
  "second_domain_accuracy": 1.0
}

maintenance_derivation_agreement and projection_rebuild_equivalence are the consistency checks: an open loop that a maintenance process derives must agree with the one the reader is told about, and a projection rebuilt from history must match the one served.

A false open is a claim about a person

The chapter’s harder point is that an open loop is a claim about unfinished work by someone. A false open is therefore not a metric artefact; it is a statement that a person has not done something they did do. That is why the suite reports false_open_rate and stale_open_rate next to recall rather than folding them into one score.

Mechanism and limitations

Documented: the suite and the metric names are in solution/run_ch9.py and the solution/open_loops/ module. The chapter’s own text names the frozen run and describes it as Type A, conditional on oracle openings.

Observed: the run above. Open-loop recall and precision were both 0.8, the false-open rate was 1.0, the false-closed rate was 0.0, and every consistency check in the suite was 1.0.

Not shown. The suite is fixture-level with zero model calls, so the open loops were supplied by the fixture rather than inferred from activity. It therefore measures the scoring and bookkeeping of a system that has been given its loops, not the ability to find them. The “second domain” row is one additional constructed domain, not an ecological sample. Nothing here shows that reporting an open loop helped anyone, or that a person agreed with one.

Canonical source and the open-loop suite .