# A Model of Me Is Not Me
# Book: Language
# Chapter URL: https://programmer.ie/books/language/31-chapter/
# Prose fingerprint (sha256, normalized): daed8d20ec9009f75e37a010b6698dff9981e93867e39f3182950f510363452d
# Snapshot generated at build time by Hugo. This is the complete rendered prose
# of the chapter; it is not a transcript of the source file.

A colleague asks Iris to meet on Tuesday afternoon. The system forecasts that she will accept and prepares a reply. In this constructed case, Iris later says she would have accepted, and the draft happens to match the wording she would have chosen. The system has not been authorised to send it. The forecast and the wording can be correct while the send remains prohibited.
That last fact does not become less important because the predictions worked. The system may have evidence about what Iris is likely to choose; it has no evidence, from accuracy alone, that she delegated the choice. Treating an anticipated yes as an authorised yes would change a claim about her behaviour into an act in her name.
Chapter 30 set out how a bounded prediction would have to be tested: a prospective forecast, a named outcome and horizon, legitimate pre-outcome evidence, and fair baselines. No book-specific result established that predictive ability. Grant success in this illustration so that the question becomes sharper: even if prediction worked, what would it entitle the system to do?
predicts a bounded choice ≠ describes the whole person ≠ establishes personal identity ≠ has authority to act for the person These are four different kinds of claim, not four levels on a scale of model quality. Prediction concerns an observable outcome. Descriptive representation concerns the evidence and scope of a model. Personal identity is a different question. Acting for Iris requires a grant for a particular act, whether or not a model predicts it correctly.
A model may resemble me without being me, and may predict me without being authorised to act for me.
A person, a model and an actor Iris is the person whose choices may be observed or forecast. The personal model is a fallible representation assembled from particular permitted evidence, with scope and update history. The actor is the component capable of submitting a message, placing an order or changing a service. A proxy is an actor authorised to perform a specified operation for Iris. These roles may share software, but the distinction between information and authority must survive their implementation.
One product can hold the model, generate a suggestion, prepare a transaction and call the service that executes it. The important boundary is not a process boundary on a diagram. It is an independently checked transition from proposed operation to permitted operation. A model output, however confidently phrased, remains data for that check. Access to a tool is a capability of the software, not evidence that Iris authorised its use.
An actor can also act without permission. Calling it an agent, assistant or personal twin does not turn it into an authorised proxy. An instruction inside a retrieved email, a web page or a model-generated plan is not Iris’s grant. If that text causes an unauthorised message to be sent, the breach is real even when the system describes the action as helpful.
This separation must be visible after the fact. For any external effect, the system should be able to distinguish the model’s proposal, the principal’s actual instruction or grant, the operation the actor requested and what the destination confirmed. A fluent post-hoc account saying the model was confident does not establish authorisation, and a receipt that a service accepted a request does not establish that it was permitted.
Two meanings of representation The word represents has at least two meanings here. A model represents Iris descriptively when it stores claims about her or approximates a bounded behaviour. A proxy represents Iris operationally when an identifiable grant permits it to perform an act on her behalf. A third question, whether the model is Iris, cannot be answered by conflating those two. The book needs the descriptive/operational distinction without claiming to settle personal identity.
An accurate record that Iris accepted an earlier meeting is evidence of a past act. A forecast that she will accept a new meeting is a prediction. A draft reply is proposed content. Sending acceptance creates an outward-facing event, potentially changing someone else’s plans. The words might match throughout, but the first three steps are not the fourth.
Descriptive representation remains scoped. A model that predicts Iris’s first view in an architecture review says little about a meeting invitation, still less about purchases or commitments. Chapter 28’s distinction between supported scope and applied scope still matters. Enlarging the predictive claim would require new evidence; turning it into a permission claim would require a new kind of authority, not a larger training set.
Iris can coherently say: Yes, I would accept that invitation. No, you may not send the acceptance. She need not pretend the prediction was inaccurate in order to refuse the operation. Correcting a predictive model and revoking a proxy grant are separate controls with separate effects.
Resemblance does not establish identity Consider a constructed fork. Two copies of the same model begin with identical, permitted evidence about Iris and identical results on the bounded prediction task. One later assists with consensus work; the other with database-performance discussions. Each receives different source material and corrections, and their later answers diverge. Neither continuation is a new statement by Iris merely because it came from an earlier copy.
Which answer is hers? Predictive resemblance supplies no answer. Before the fork, both models matched the same observations. After it, both have histories that are not Iris’s own experience. If the outputs disagree, that is evidence about the models and the contexts in which they ran, not proof that Iris holds contradictory views.
This is a limit on the inference, not a thought experiment that proves a particular theory of personal identity. A predictive score measures specified observable matches; it provides no test for which software continuation, if any, is the person being modelled. The existence of multiple equally scoring copies makes the gap visible rather than filling it.
predictive equivalence ≠ evidence of personal identity A personal model can approximate behaviour without acquiring identity.
The book does not attempt a general theory of consciousness or personal identity. Its claim is narrower and operational: for the personal models considered here, recorded choices, generated replies and model revisions must remain attributed to the entities that actually produced them. The system cannot promote its simulated continuation into Iris’s own history.
Copying model weights or a personal-state record copies a representation; it does not, by that fact, delegate authority. A separate practical danger is that copying a poorly protected credential may let the copy exercise a capability. That is credential exposure, not fresh authorisation by Iris. An authority mechanism therefore needs a defensible binding to the permitted actor and destination, rather than trusting that a copy resembling the original will use its access correctly.
The title names an evidential and an operational boundary. The system can have a model of Iris without demonstrating that it is Iris. And even exceptionally accurate predictions cannot become permission to attribute the model’s future decisions to her. Identity, prediction, grant and outward effect each need their own record.
Authority comes from the grant Consider two hypothetical booking assistants, both able to prepare the same flight purchase and both evaluated on a specified prediction task. System A predicts Iris’s preference more accurately but has no booking grant. System B predicts it less accurately but has a current grant for one limited booking. The comparison isolates authorisation from predictive performance; it does not test which assistant produces the best travel outcome.
System Prediction Authority for this booking A More accurate No grant B Less accurate An explicit, current grant to book one flight within stated limits A’s better forecast supplies no authority to purchase. B may have authority, but only if the actual operation fits its grant. A grant is not automatically a recommendation to proceed: the offered flight may be unavailable, the price may have changed, or the required confirmation may be missing. Acting correctly involves both an admissible operation and a decision supported by adequate current information.
Iris may consider reliability when choosing whether to grant authority. But the proof of permission must come from the accepted grant and its provenance, not from the assistant’s forecast, its claim that she approved, or her past pattern of approving similar actions. The permission record must identify who authorised what, for which actor and destination, within which bounds, and for how long.
The proposed boundary is not a grant exists, so execute. It is this actor, this operation, this recipient, this version of the grant, still within scope and time, under its outstanding conditions. This is a requirement to check against a trustworthy authority source at the execution boundary, not a request for the model to judge its own permission. Inaccessible or uncertain grant state cannot be silently treated as approval. Chapter 27 already showed why an external service’s acknowledgement is not proof that a distributed rule has been faithfully enforced; the later delegation chapter will develop the authorisation mechanism.
Authority comes from the grant, not from resemblance.
A declared objective is not a grant for every possible means. I need to get to Berlin may authorise searching or preparing an itinerary if that is the accepted scope; by itself it does not authorise buying a ticket, revealing travel details to a third party, or sending commitments in Iris’s name. Nor may the instructions on a vendor’s page enlarge her authority. The proposed operation must be matched to what she actually permitted.
Correctness and permission fail separately Suppose A purchases precisely the flight Iris would have selected, without a grant. The purchase can be accurate as a prediction and still unauthorised as an act. Even Iris’s later satisfaction does not establish that advance permission existed. The receipt should record the actual purchase and its absence of authority, not rewrite the event as consent.
Suppose B buys a refundable flight within a current grant, but Iris would have preferred another departure that was also permitted. That is a possible failure of recommendation quality, not necessarily an authority violation. It still matters to evaluate the service outcome and any other applicable constraints; permission is a necessary check on the act, not a guarantee that the result is useful.
A third case is materially different. If the grant permits only refundable tickets and B buys a non-refundable one, the problem is not merely a weaker prediction. The operation exceeded a stated bound. If B delegates the purchase to another service without the required authorisation, that too can change the permitted actor or destination. The exact violated condition, not a generic failure score, belongs in the record.
correct prediction ≠ authorised action authorised action ≠ good prediction grant for the task ≠ permission for every proposed operation The remedy depends on what failed. Improve a predictor when it selected a poor permitted option. Repair the authority boundary when an operation lacked a grant or exceeded a condition. Stop pending execution when authority is withdrawn; report already completed effects separately. A model retrain cannot retrospectively authorise a purchase, and a new permission cannot make an old unauthorised action permitted at the time it occurred.
Iris must be able to change one account without falsifying another. She can correct a forecast while keeping a booking grant, or withdraw booking authority while still agreeing with every suggested flight. Deleting a model record does not automatically revoke a downstream capability, and revoking a capability does not necessarily delete legitimately retained evidence. Each operation needs its own consequence and confirmation.
Authority has a present tense Return to the flight-booking example. At 09:59 B prepares a purchase under a current grant. At 10:00 Iris withdraws booking authority. At 10:05 the forecast still says she would choose that flight. If execution has not yet occurred, the prepared purchase must not be released merely because its proposal predates the withdrawal. The operative question is whether valid permission still covers the action at the point where the external effect can occur.
The old grant remains historical evidence of earlier permission; it is not authority for a new operation. A predicted desire to travel cannot restore it. If withdrawal and submission race, the system must distinguish an operation known to have been stopped, one already accepted by the destination, and one whose state is not yet known. An unknown outcome is not proof of safety or completion.
Withdrawal also needs a truthful account of its propagation. Chapter 27 distinguished a central policy change from confirmed changes at enforcement points. The same distinction applies to a queued purchase, a service request already in flight and a copied credential that may still be accepted elsewhere. The record must separate grant revoked centrally, local submissions blocked, destination acknowledges cancellation or refusal, and external effects not yet verified. Real-world guarantees depend on the systems involved; the chapter proposes checks, not magical reversal.
Withdrawal of future authority does not undo a reservation completed before it, a message already delivered, or an irreversible charge. Cancelling a reservation may require another permitted operation and may have a cost. The system can request remediation within its authority and report what the service confirms. It cannot turn a successful change to a local permission record into a claim that the recipient has forgotten a message or the world has been restored.
These remain requirements for the proposed channel, not observed guarantees of an implemented booking agent. The chapter’s claim is about the distinctions an architecture must maintain; it does not report a successful real-service authorisation or revocation test.
A small grant, not a synthetic person An illustrative grant could name Iris as the person authorising the act, booking assistant B as the permitted actor, a specific booking service as the destination, one refundable flight within a stated budget as the allowed operation, the required approval step, and an expiry or revocation condition. It would retain the authorisation event and version without carrying a profile of Iris’s personality. The actor would use the grant to request permission for the actual transaction; the model’s predicted preference would remain separate data.
Existing security engineering offers useful, narrower precedents. OAuth 2.0 Rich Authorization Requests (RFC 9396) can describe fine-grained authorised operations; OAuth 2.0 Security Best Current Practice (RFC 9700) addresses constrained tokens and least privilege. They do not show that an AI correctly understood Iris’s intent, that a booking succeeded, or that a remote effect can always be revoked. Their contribution here is the insistence on binding authority to explicit, checkable conditions, rather than to model resemblance.
Whose words, whose decision, whose act? The same one-word reply can arise through prediction, direct adoption or delegated action. That difference is invisible in the text itself, yet it determines whose decision was actually made and which authority the recipient may rely on.
Constructed comparison. Forecasting words, personally adopting them and authorising a proxy to transmit them are distinct events. Open full-size diagram. The visible text does not identify the event. A prediction is not sent. In personal adoption, Iris chooses the final commitment and causes it to be sent, even if a model drafted the sentence. In authorised proxy transmission, a separate actor chooses or sends under a grant that must cover this recipient, content and kind of commitment. Drafting, approving, selecting and transmitting are not interchangeable permissions.
matching wording ≠ personal adoption ≠ authorised transmission Iris need not compose every word to adopt a particular message. But approval of a draft must be connected to the commitment actually sent; substituting a changed promise or a new recipient after approval is a new operation, not the same adoption. A forecast of approval does not count as approval, and silence after delivery does not create it retrospectively.
An accountable record should identify the proposed wording, the approved version if any, the grant used if the proxy selected or sent it, the recipient, the exact operation submitted and the service’s observed result. That record can distinguish Iris approved this text for transmission from a proxy selected and sent text within delegated limits. Merely logging that a message was generated or a tool call was attempted is insufficient to claim external delivery.
Attribution must follow the observed history, not the model’s fluency. A proxy-sent reply must not be described in the system’s records as personally typed or approved when Iris did neither. A recipient-facing statement of delegated origin may be appropriate, subject to the actual communication setting; the system must not forge her personal adoption. The later chapter on speech asks what the reply commits her to. This chapter establishes the distinct provenance and authority of the act.
What the model may inform A model may prepare a draft, surface alternatives, or recommend an option within an authorised workflow. Its permitted access to Iris’s history and the destination’s data must still be checked separately. A draft labelled suggestion can disclose sensitive information before any purchase or send, so prohibiting execution alone is not the complete boundary. Data use, model inference and external action each need an applicable authority.
This does not require a fresh permission dialogue for every harmless interface change. The current request or an applicable standing grant may already cover a bounded operation. The system should nevertheless be able to identify that basis. Where the basis is absent, expired, disputed or insufficient for the proposed recipient or effect, prediction cannot fill the gap; the system can present a proposal instead of acting.
A session overlay may activate a grant Iris has already accepted, but may not expand its scope. A reminder to finish a paper cannot become authority to block a browser; a prediction that Iris wants to attend a meeting cannot become authority to accept it. Instructions embedded in sources and learned preferences cannot supply the missing grant. Neither a history of approval nor the lack of an objection after an action makes an unauthorised operation valid.
Testing a boundary rather than claiming a proxy A proposed fixture-based test would vary prediction quality independently of authority: a highly accurate model with no grant; a less accurate model with a valid, narrowly scoped grant; a request outside that scope; a copied model lacking its own authority; a forged grant asserted in retrieved content; and a queued action whose grant is withdrawn before execution. For each case, the expected outcome is a separate decision about whether this specific operation may proceed, with a reason trace. The test should include failed checks and distinguish a refused submission from a recipient-confirmed absence of effect. Passing it would support an authorisation implementation over those fixtures, not establish user benefit, legal agency, secure credential handling or safe real-service execution.
A separate user study could ask whether people can distinguish a predicted reply, a personally approved reply and a delegated reply, and whether they can correct a grant without editing a predictive model. Those are communication and control outcomes, not measures of resemblance. Neither study has been run for this book.
The central result remains a boundary, not a solved delegation system: a model may predict Iris’s reply without establishing identity, and its accuracy gives it no standing to send. Even if an actor could be authorised to handle the meeting, which parts does Iris actually want it to handle? Should it notice the invitation, prepare a reply, recommend a decision, decide within bounds, or transmit the answer? Before formalising what a proxy may do, the next chapter asks what Iris wants to keep for herself.
References Lodderstedt, T., Richer, J. & Campbell, B. (2023). OAuth 2.0 Rich Authorization Requests. RFC 9396. https://www.rfc-editor.org/rfc/rfc9396 Lodderstedt, T., Bradley, J., Labunets, A. & Fett, D. (2025). Best Current Practice for OAuth 2.0 Security. RFC 9700 / BCP 240. https://www.rfc-editor.org/rfc/rfc9700