Tell a current fact from a historical one

“Is it still true?” is two questions, and a memory that answers the wrong one is worse than one that answers neither. The suite here scores current state accuracy and historical state accuracy separately, per temporal shape, so a system that remembers everything and a system that only remembers the present are not averaged into one number.

Run it

git clone https://github.com/ernanhughes/project-memory.git
cd project-memory
git checkout 0a6419ab0d6f65e491374b971d214255f2b0d8e8
python solution/run_ch8.py --outdir ./runs/ch8-demo

Python 3.11 or later. The suite is fixture-level and deterministic: zero model calls, no database, no service. It writes a new run directory; frozen runs are never overwritten.

Expected outcome

Five temporal shapes, and the two questions scored apart:

{
  "current_state_accuracy": {
    "T0-bag": 1.0,
    "T1-arrival": 1.0,
    "T1b-event-time": 1.0,
    "T2-ordered": 1.0,
    "T3-temporal": 1.0
  },
  "historical_state_accuracy": {
    "T0-bag": 0.0,
    "T1-arrival": 0.0,
    "T1b-event-time": 0.0,
    "T2-ordered": 1.0,
    "T3-temporal": 1.0
  },

The system here is current-only: it gets every current-state question right and fails every historical one, except where the shape is ordered or temporal and history is recoverable. That is the shape of the failure the chapter is about, and it would be invisible in a single averaged score.

The rest of the suite is the checklist around that distinction:

  "temporal_role_accuracy": 1.0,
  "order_sensitive_change_accuracy": 1.0,
  "order_invariant_stability": 1.0,
  "late_arrival_robustness": 1.0,
  "future_effective_accuracy": 1.0,
  "correction_revision_accuracy": 1.0,
  "partial_order_calibration": 1.0,
  "sequence_gap_detection": 1.0,
  "projection_rebuild_equivalence": 1.0,
  "temporal_causality_violation_rate": 0.0,
  "supersession_accuracy": 1.0,
  "valid_time_accuracy": 1.0,
  "transaction_time_accuracy": 1.0
}

Two clocks, two questions

The two accuracy families are not the same measurement. current_state_accuracy asks what is true now; historical_state_accuracy asks what was true then. A memory that keeps only the present can score 1.0 on the first and 0.0 on the second, and a memory that keeps everything without distinguishing the two can score badly on the first while looking well-informed.

projection_rebuild_equivalence is the check that the two stay consistent: a projection rebuilt from the stored history has to match the one the system serves. A system can be right about the present for the wrong reason and still fail here.

Mechanism and limitations

Documented: the suite, the five temporal shapes and the metric names are in solution/run_ch8.py and the solution/temporal_memory/ module. The frozen run for this chapter is named in the chapter’s own text, and a run manifest records the corpus, the seeds and the digests of everything the result depends on.

Observed: the run above. The current-only system scored 1.0 on all five current-state shapes, 0.0 on the three historical shapes where history is not recoverable, and 1.0 on the ordering and temporal shapes where it was. The temporal role, supersession and both clock accuracies were all 1.0, and the temporal causality violation rate was 0.0.

Not shown. The suite is fixture-level: it checks a system against constructed temporal shapes, not against a real memory under real traffic. A score of 1.0 on a fixture is evidence that the specific failure mode does not occur in that fixture, and nothing more. The run records what it measured, not whether any of it helped a person. order_invariant_stability and partial_order_calibration are proxies for properties a reader would experience, and the suite does not check that the reader did.

Canonical source and the temporal suite .