# How Much of You Can an AI Reproduce?
# Book: Language
# Chapter URL: https://programmer.ie/books/language/30-chapter/
# Prose fingerprint (sha256, normalized): 0a215ef0ff19c9c8a93bb97eabfdd9433bf37fc2a9cd86b7af2d48d395a00d0f
# Snapshot generated at build time by Hugo. This is the complete rendered prose
# of the chapter; it is not a transcript of the source file.

Iris opens a new architecture review. Before the screen appears, the system forecasts that she will choose the expanded calculation rather than the summary. It has a reliable record that she chose the calculation in the previous review. That record establishes something about the previous review. Whether the new forecast succeeds depends on a choice she has not yet made, under presentation conditions the system must also record.
The distinction is small and easy to miss in a fluent interface. “You chose this last time” becomes “you will choose this now”, then “I know how you think”, and finally “I can answer for you”. Those statements make claims about history, prediction, understanding and authority, respectively. The second does not follow from the first. The third does not follow from the second. Even an accurate forecast gives no permission for the fourth.
Part III asked how little personal state good communication requires. Part IV begins with a different question:
How much of a person’s observable behaviour can an AI predict, and what would count as evidence that it can?
The title names the larger temptation. This chapter therefore begins with a deliberately bounded test. The system forecasts a specified observable choice, records the forecast before the outcome is known, scores it on genuinely held-out cases (cases the predictor was not built or tuned on), and compares it with simple alternatives given the same current context. Iris’s scenes and all numerical examples below are constructed illustrations. They are not measurements of a real person or of the book’s reference implementation. Part IV cannot earn a claim about a personal proxy merely by giving that proxy a name.
Four different claims The claims need separate names before they can be tested separately.
RECALL what the person previously said or did ESTIMATION an inference about something presently unobserved PREDICTION a forecast of a specified future observable outcome REPRODUCTION a claim to reproduce behaviour across a stated range of situations Recall retrieves an earlier choice. Estimation might infer which explanation Iris currently prefers when she has not declared it. That is a claim about an unobserved present condition. Prediction commits, in advance, to a later observable choice. An estimate may eventually be checked against independent evidence, but that does not change the time to which its claim refers. None of these operations licenses inferences about mood, stress, fatigue or motivation. Nor do they license keeping observations that were not authorised for that purpose.
Reproduction is a broader claim about behaviour across a declared range of situations. That range includes how choices change when tasks, information or instructions change. A reproduction test must state whether the system is merely forecasting Iris’s responses, generating a sequence of choices in her place, or interacting with an environment that also reacts to it. Matching several interface preferences does not automatically demonstrate any of these broader abilities. A bounded behavioural benchmark cannot decide identity, subjective experience or moral standing. It can only measure the behaviours it actually tests.
These distinctions expose two misleading tests. If an answer is already in an accessible history, recovering it demonstrates recall, not prediction. If a model generates plausible responses in Iris’s voice, it may demonstrate stylistic imitation, not an independent forecast of a held-out decision. In both cases, the evaluator must know whether the outcome, a paraphrase of it or a derivative record was present in the inputs.
Name the outcome “Predicts Iris” is too large to score. “Predicts which of three equally available views she selects first in an architecture review” has a defined outcome. So does “predicts whether she explicitly requests the derivation before leaving that review.” These are different targets and should not be pooled without saying how. The study must define the offered alternatives, their position and default setting, what counts as a selection, and what happens when Iris makes no selection or the session ends without a recorded outcome.
The prediction also needs a context-and-horizon contract. The contract fixes what is being forecast, under which conditions, and over what period:
PREDICTION CONTRACT target: first selected view: summary / table / derivation / none context: architecture review of a familiar subject; no stated urgency horizon: first eligible review within seven days interface: same visible choices, order and default for every method evidence: only permitted records available at forecast time cutoff: forecast sealed before the selection is observed scoring: predefined treatment of no choice, abstention and missing outcome “Iris will choose the terse explanation” leaves these boundaries unstated. A forecast about the first eligible ordinary review cannot be relabelled as a forecast about the next incident six months later. If no eligible review occurs within seven days, the trial has no observed target outcome. It must be reported as such, not scored as a correct abstention and not silently replaced with an easier review.
prediction error ≠ person changed ≠ context changed These possibilities can overlap. If the forecast for an eligible review is wrong, it counts as a miss under its prespecified rule, even if a change in context later helps explain it. Changed context must not become a retrospective exemption. If a new explicit instruction makes a review ineligible under the original contract, that classification needs a rule set in advance and a recorded reason. The experiment must still account for excluded and unobserved cases. A mistake does not prove that the person changed, and a changed choice does not prove that a memory of the earlier choice was false.
The target matters because an observable action is only a proxy for what its label may suggest. A click can reflect the position of a button, the default view, the wording of an option, a requirement of the task or simple convenience. Not requesting detail does not establish understanding. Choosing a summary does not establish an enduring preference for summaries. If the question is whether the reader understood a qualification or reached a sound decision, those outcomes need separate evidence and a study design that measures them directly.
The same restriction applies to reasons and values:
predicting a choice ≠ predicting a reason ≠ predicting a value ≠ reproducing a person A system might correctly predict the table because the table is the most frequently chosen view. That does not mean it knows why Iris selected it. Predicting a reason she later reports is a different outcome, and it still does not establish access to her private reasoning. Inferring a stable value from a handful of view choices goes further again. The value, the contexts in which it applies, the contrary cases and the evidential standard would each have to be defined separately. Galit Shmueli’s distinction between explanatory and predictive modelling is useful here. A predictor can work well without establishing an explanation of the process that generated the choice.
Scope expansion is a new inference, as in Chapter 28. Success at predicting the first view in a technical review cannot be renamed success at predicting Iris’s decisions in general. Even within a review, selecting a view, understanding it, finding it sufficient and deciding what to do are different observable outcomes. The ambition of the claim must not exceed the measurements available to support it.
Hold the future out The essential boundary is temporal. What was available when the forecast was made? What became known afterward?
For a prediction of Iris’s next selection, freeze the authorised input snapshot, the model or decision-rule version, the permitted adaptation state and the forecast, all before her choice is observed. Record both when an event happened and when its information became available to the predictor. A correction written afterward cannot travel backward into the input, even if it refers to something that happened earlier. Rewriting a prediction after seeing the outcome tests hindsight, not foresight.
Held-out cases must be genuinely new for the claim being tested. A different file containing the answer, a transcript summarising the event, or a near-duplicate of a case already reviewed can leak the target across an apparent split. (Leakage means information about the answer reaching the predictor through a route the test was meant to exclude.) For this chapter’s forecast claim, a prospective chronological evaluation is preferable. Establish the rule from earlier permitted cases, seal a prediction, and then score later cases in order. If the system may learn between forecasts, the update rule and its access to previous outcomes must also be fixed. A frozen-model test and an online-updating test are different experiments.
The record should make the target, task conditions, eligible population, allowed inputs, time cutoff, forecast, any abstention, observed outcome and scoring decision inspectable. Record enough to audit leakage and reproduce the comparison, but not every detail of Iris’s activity. Study participation and permission to use historical records for prediction must be authorised separately. Collecting more personal history is not justified by its apparent usefulness to the metric. A later request to withdraw data also needs a defined treatment in stored experimental artifacts.
Conceptual guide to the chapter; no new empirical result is asserted. Open full-size diagram. Compare with the plain alternatives Correct predictions are only the beginning. Perhaps most people select the expanded calculation in this kind of review. Perhaps Iris simply repeats her last selection. A personal predictor has to show what it adds beyond those simpler explanations.
At minimum, the score needs these two prespecified comparisons:
Generic default: a declared rule that does not use Iris’s individual history, such as the common choice for this task, established without the held-out answers. Repeat-last-choice: the last legitimately available choice in a comparable task, where “comparable” is defined before the new outcome is seen. When there is no prior comparable choice, the repeat-last-choice baseline needs a declared fallback. Report how often that fallback was needed, and keep those cases in the eligible population. A personal model must not be evaluated only after it has built up a useful history while the non-personal baseline is judged from the first interaction. If the amounts of evidence available to each method differ, report that difference as a condition rather than hiding it in the result.
The comparisons also need fair access to current task information and to the same choice interface. If Iris explicitly declares that she has five minutes, the generic task-based rule may use that declaration. The personal predictor should not receive better task descriptors, later metadata, a different option order or a stronger default. Otherwise the experiment cannot attribute any gain to the retained personal history, rather than to a more informative present or a more persuasive interface.
A predictor that ties the baselines may still sound persuasive, but the tie does not show that retaining Iris’s history improved the forecast. An advantage should be reported as a difference with a justified uncertainty estimate, not only as two rounded percentages. Repeated choices from one person are related observations. Uncertainty calculations should respect sessions and their temporal dependence rather than treating every click as an independent sample. Even a genuine advantage supports only the defined target and population.
A probability is a different kind of forecast A forecast can name one choice, or it can assign probabilities to the available choices. “Summary: 0.70; table: 0.20; derivation: 0.10” makes a more specific claim than simply naming summary. It lets the evaluator ask whether confidence tracks observed frequencies. A forecaster that assigns 90 percent confidence to almost every decision, and is right only seven times in ten, is miscalibrated. This holds even if its top choice often beats the baseline.
If probabilistic forecasts are part of the experiment, use a prespecified proper scoring rule, such as the Brier score or log loss. A proper scoring rule rewards forecasts that report the probabilities the forecaster actually believes, so it penalises both overconfidence and vagueness. Keep top-choice accuracy as a separate outcome. Inspect calibration only with enough data to make it meaningful, and report the sample size rather than treating a confidence label as a measured probability. A system that abstains must still account for coverage and for the forecasts it declined to make. These are established tools of probabilistic prediction. They are not evidence that the system proposed in this chapter emits calibrated forecasts.
Compare Iris with herself, carefully There is a third, informative comparison: Iris’s own retest consistency. Ask comparable questions on separate occasions and measure agreement. The interval, option layout, preceding information and number of paired observations must all be made explicit. Repeated questioning can itself teach, fatigue or anchor a reader, so a retest is not a neutral window onto a fixed person. The purpose of the comparison is to describe stability under specific conditions, not to uncover an unchanging inner answer.
This measures agreement between two of her responses. It does not show Iris forecasting her own future response, and it does not set a ceiling for every predictor. A predictor with legitimate current information might anticipate a change that repeating her old answer would miss. Repeated responses can also include transient mistakes. The comparison is meaningful only when the target, population, timing and conditions are comparable.
Retest agreement gives context for a predictor’s score. It is neither a proven ceiling on predictability nor a substitute for a baseline. A model with legitimate new context might anticipate a change that repeat-last-answer cannot. Dividing a predictor’s accuracy by retest agreement does not produce a valid percentage of “Iris reproduced”. Show the paired counts, the conditions and the disagreements, including whether the answer options themselves changed.
For a forecast of her later choice, that later choice is the observed label under the declared interface conditions. Her earlier answer can still be an accurate historical record even when using it for prediction fails. Conversely, a changed answer does not show that she is inconsistent as a person. Chapter 24’s distinction between historical truth and current applicability survives unchanged in this measurement framework.
Coverage is not accuracy A predictor may decline to forecast where it has insufficient evidence. That creates a second quantity to report. It is not a way to make difficult cases disappear.
Suppose there are one hundred eligible cases. The system predicts twenty of them and gets nineteen right.
Quantity Constructed count What it says Coverage 20 of 100: 20% How often the system supplies a prediction Accuracy among predictions 19 of 20: 95% How often its supplied predictions are correct Correct predictions across eligible cases 19 of 100 How much of the eligible set has a correct forecast Abstentions 80 of 100 Where it deliberately supplies no forecast The eighty abstentions establish neither eighty correct predictions nor eighty errors. They establish only that the model left most eligible cases without a forecast. Report the selective accuracy (accuracy on the cases where the system chose to predict) and the coverage together, along with the full eligible count and the outcomes that were not observable. A headline of “95% accurate” without “on 20% of cases” is an incomplete description of this constructed result.
The abstention rule must operate before the outcome is known. Score the baselines on the same attempted cases and on the full population, and compare methods at comparable coverage. A curve showing error or accuracy against coverage is more informative than a single favourable operating point. If the method uses a confidence threshold, its calibration must be assessed on held-out forecasts. The threshold does not become trustworthy simply because it is written into a policy.
Deliberate abstention is not the same as a crashed prediction, an invalid output, a user who declined to participate, or a case whose eventual choice was never observed. Keep separate counts for each, along with the rule for including or excluding them from the accuracy and coverage denominators. In particular, an unobserved outcome cannot be treated as a safe or successful prediction merely because the system would have abstained.
Abstention also changes the interaction. Iris may receive a generic default, choose for herself, or be asked an additional question. That cost belongs in the evaluation. A model that forecasts rarely and accurately may still impose more work than a stable, reader-controlled interface. Conversely, avoiding a damaging wrong default can be valuable. The comparison should measure these consequences rather than assuming that coverage or accuracy alone represents usefulness.
Keep changed contexts visible Return to a smaller constructed case. Over ten reviews, the personal predictor gets eight first-view choices right. The generic task default also gets eight right, and repeating Iris’s last comparable choice gets eight right. In this illustration, all three succeed on the same routine cases and miss the same two changed cases.
One changed case is a review before a meeting, with five minutes explicitly available. The other is an incident investigation whose declared immediate goal is finding the failure. Under the constructed scoring rule, all three methods miss both. This shows no gain from personal history over the baselines on these ten examples. It does not demonstrate that the methods had identical inputs, nor that a context-aware baseline would necessarily fail. With the declared objective supplied, such a baseline might make a different prediction.
The aggregate also hides a distribution: eight successes on eight routine cases and no successes on two changed cases. These are intentionally tiny, illustrative counts, not a measured population and not an estimate of future error. An abstaining predictor could report perfect accuracy on the routine cases and zero coverage on the changed ones. Report both quantities by prespecified context group, and resist drawing a new general rule from two difficult examples.
Relevant conditions include the task, available time, device, explicit mode and declared objective. They need not be guesses about fatigue or motivation. Register the context groups and the inputs used for each forecast before scoring. Exploratory examination of errors may suggest new conditions, but those hypotheses need another held-out test. This matters especially when a small number of changes supplies most of the interesting narrative.
Reporting by context also guards against treating an old preference as a permanent target. If Iris revises her standing instruction, prediction should be assessed against the behaviour actually being forecast, with the time and scope of the revision visible. Success under the previous policy does not demonstrate success after it changed.
State the conditions of the prediction The interface itself can alter the outcome. Suppose the predictor expects Iris to choose the summary and therefore places it first or makes it the default. The observed click is then a response to that policy, not an independent measurement of what she would have chosen under a fixed presentation. The predictor may even appear to improve because its defaults steer the labels later used to retrain it. Recommender-systems research describes related feedback effects as preference-data pollution. That literature motivates the concern but does not measure this book’s interface. Record the exposure and the interface policy for every choice.
Behaviour observed under a temporary overlay carries that provenance, as Chapter 25 requires. A forecast made during a five-minute review cannot quietly become evidence that Iris has a lasting preference for brevity. The system also must not create the permission it needs by placing a prediction into the interface and then treating the resulting click as freely given consent. Measurement does not suspend the rules governing observation, retention, derived state or future use.
Sequences introduce another boundary. In teacher-forced evaluation, the predictor receives Iris’s actual preceding choices before predicting each next one. In a free-running sequence, the predictor proceeds using its own earlier generated choices, subject to whatever new external information the protocol allows. These are different tests. Errors can compound in the second, and the later situation may itself depend on the previous choice. Specify whose actions alter the environment and how the environment responds. Multiplying isolated accuracies together would require independence assumptions that this chapter has not established.
pointwise predictive accuracy ≠ trajectory fidelity Three correct isolated choices do not show how Iris moves between decisions. A new piece of evidence may lead her to revise her view. That revision may alter the next question and the choices available afterward. A bounded reproduction test needs to score sequences, dependency handling, revisions and changed context, not just matching final answers. It must also say what the model does when it abstains or encounters a situation outside its authorised scope.
These distinctions prevent “predicts her behaviour” from concealing what was forecast, under whose interface, with which previous choices and which information. Report the number of people and sessions, the contexts and horizons, the coverage, the uncertainty and the gap to each baseline. A within-Iris result does not generalise automatically to other readers. An average across many readers does not establish that any particular reader is predicted well. Neither kind of evidence establishes a broad personal replica.
Prediction is not communication benefit The book’s interest remains communication. A small, scoped predictor might help choose a first view without attempting to reproduce a person. That is a possibility to test. It is not a result earned by a prediction score.
A prospective test the book could actually run The first experiment should be smaller than a claim to reproduce Iris. With informed participation, recruit several readers for repeated, comparable technical-review tasks. Freeze the available presentations, the current-task facts, the choice target, a time cutoff and a stopping rule before collecting evaluation outcomes. Collect only the records participants have authorised for prediction. Compare the generic task rule, repeat-last-choice, and a bounded personal predictor on the same prospective cases.
First, evaluate forecasts without using them to change the interface. All participants see the same options in the same positions. This separates forecasting ability from the effect of a policy on behaviour. In a separately authorised intervention study, compare a fixed first view, a reader-selected first view and a predictor-selected first view, using appropriate assignment to conditions. Measure comprehension, preservation of critical qualifications, time, corrections and perceived control, rather than clicks alone. A forecast-only study and an intervention study answer different questions.
Prespecify the minimum evidence needed before declaring an improvement, how repeated sessions will be handled, the treatment of abstentions and no-response cases, and checks for policy changes during data collection. Report adverse effects and null results, including cases where the generic or reader-controlled alternative wins. The experiments described here are proposals. This chapter reports no reader study, no personal forecasting benchmark and no production Pi prediction run.
The communication question needs independent outcomes: fidelity to the source, comprehension of essential qualifications, decision quality on an appropriate task, attention cost and ease of correction. Predicting which view Iris will click may improve none of these. A seldom-selected view could be safer if it preserves a condition that the selected view omits. Comparing interfaces on these outcomes should keep preservation, accessibility, permission and the available choice set fixed.
communication benefit ≠ predictive accuracy ≠ behavioural reproduction ≠ identity ≠ authority More personal evidence is not justified merely because it increases prediction accuracy. Collecting and using it requires legitimate authority, and its cost includes retention, privacy exposure, review and correction. The relevant design comparison is not “personal predictor versus nothing” but “personal predictor versus a competent, task-aware default and an interface Iris can control”. Part III’s requirement to justify additional state carries over into prediction.
The defensible claim at the end of this chapter is bounded. For a declared target, interface, population, context and horizon, with permitted evidence frozen at the forecast cutoff, the system produced a particular accuracy and coverage, with these failures, these uncertainty estimates, calibration results where probabilities were supplied, and comparisons with these baselines. A reproduction claim would require more: behaviour across a prespecified range of situations and sequences. None of the chapter’s constructed counts demonstrates such a result for the reference implementation.
Now suppose a predictor eventually performs well on the narrow target of Iris’s responses to meeting requests. A colleague asks about Tuesday afternoon, and the system forecasts acceptance. That forecast can be scored only after the relevant response is observed, under the recorded conditions. Even a consistently accurate forecast says nothing about whether the system has permission to send the acceptance. Chapter 31 begins at that boundary: a model of Iris is not Iris, and a likely answer is not an authorised answer.
References Shmueli, G. (2010). To explain or to predict? Statistical Science, 25(3), 289–310. DOI 10.1214/10-STS330. (For the distinction between explanation and predictive performance.) Gneiting, T. & Raftery, A. E. (2007). Strictly proper scoring rules, prediction, and estimation. Journal of the American Statistical Association, 102(477), 359–378. DOI 10.1198/016214506000001437. (For scoring probabilistic forecasts, not for evaluating this book’s system.) Adomavicius, G., Bockstedt, J. C., Curley, S. P. & Zhang, J. (2022). Recommender systems, ground truth, and preference pollution. AI Magazine, 43(2), 177–189. DOI 10.1002/aaai.12055. (For the risk that recommendation interfaces influence the preference observations used as evaluation data.)