Too Much Information

Return to the incident manager from the last chapter, now with a Semantic Browser that works. She opens the report on Tuesday’s outage, and the system has found a good deal more than the report. There are forty earlier incidents that look similar. There is service telemetry for the hour in question. There is documentation for three dependencies, and a runbook that has not been updated since the migration. The record has turned up three possible contradictions between the report and other sources. The system has drafted seven diagrams of the causal story, each from a slightly different angle. There are fourteen relevant excerpts from the team’s chat, the change history for the week, and a handful of external write-ups of the same failure mode.

Every one of these might be useful. Showing all of them would wreck the interface. The browser has done what Part I promised: it can find, derive and generate a great many plausible things to put in front of a reader, cheaply and quickly. And that has created a new problem, which is the subject of this part. Once a system can produce more potentially useful communication than the receiver can attend to, which of it deserves to reach her?

PART I
representation was the problem
        ↓
representation becomes cheap
        ↓
PART II
selection becomes the problem

For most of history, the scarce thing was the representation itself. Someone had to write the summary, draw the diagram, find the related paper. Whatever reached the reader had been through the filter of that effort. Now the effort has largely gone, and the filter with it. What used to be constrained by cost has to be constrained by judgement.

From abundance to a ladder of decisions

It is tempting to say there is simply too much information, and to leave it there. But there has always been too much information. What is new is that intelligence in the channel can manufacture candidates for relevance faster than any receiver can inspect them. To see what has to be decided, it helps to separate steps that ordinary talk runs together:

what exists
   ≠ what is relevant
   ≠ what should be selected
   ≠ what should be shown
   ≠ what will be noticed
   ≠ what will be useful

Laid out as a ladder, with the incident manager’s material:

AVAILABLE   something could be accessed
               the forty incidents, the telemetry, the chat history
     ↓
ELIGIBLE    it satisfies basic constraints
               she is allowed to see it, it is current, it is about her service
     ↓
RELEVANT    it bears on the current task
               the three that match the failure she is deciding about
     ↓
SELECTED    the system chooses it
               two incidents, one contradiction, one diagram
     ↓
PRESENTED   it enters the representation
               shown in the view, or placed in the margin
     ↓
NOTICED     the receiver attends to it
               she actually looks at it
     ↓
USED        it changes how she reconstructs the situation, or what she does
               she changes her decision

Each rung loses something, and each loses a different thing. Available material is cut by what the system can reach and what is permitted. Eligible material is cut by what bears on the task. Relevant material is cut by the budget of space and time, since not everything relevant fits. Selected material may be presented in a form that is easy to skip. Presented material is only sometimes noticed. And noticed material is only sometimes useful. A system that improves one rung can leave the others untouched, or worse.

Three of the steps deserve to be stated as separations, because the pull to merge them is strong:

available  ≠  selected  ≠  noticed  ≠  useful

The first is the system’s decision against the world, the second against the receiver’s attention, and the third against the receiver’s purpose. Notice that the last two stages are on the receiver’s side. The system can choose and present. It cannot make anyone notice, and it cannot make what they notice useful. Every decision it makes upstream is a bet about what will happen downstream, in a mind it cannot observe.

Abundance is not overload

It helps to be exact about what the problem is, because “information overload” is a lazy phrase. Abundance and overload are different things.

information abundance  ≠  information overload

Abundance is a property of the environment: there is a great deal that could be accessed. Overload is a mismatch among several things at once: the information available, the capacity of the receiver, the task in hand, the time there is, and the cost of inspecting what is offered. Take any one of those away and the same environment is not overloaded. Give a reader a week and she can read the forty incidents. Give her a task that needs exactly one of them and the other thirty-nine are not a burden. Sometimes another piece of information is precisely what she needs. Sometimes one more annotation ruins the page.

So the problem is not how to minimise information. A system that shows as little as possible is just as capable of failing her, by withholding what would have changed her mind. The problem is allocation: how to share a scarce channel among many competing communicative possibilities, each of which might be good.

Herbert Simon put the economics sharply in 1971. A wealth of information, he wrote, creates a poverty of attention, and organisations, and now individuals, have to decide where to spend it. It is worth being careful not to read “attention” as a purely psychological quantity. What is scarce is wider than the moment of looking. It includes screen space, the cost of inspecting an item to see whether it is worth anything, the time in a working day, and the number of times a person can be interrupted before the interruptions stop meaning anything. These are all parts of one finite channel between the system and the person.

Three things a system can do with a good candidate

A candidate that has reached the stage of being worth considering can be handled in at least three ways, and they are different communicative acts:

SHOW NOW        put it in front of the receiver
MAKE AVAILABLE  keep it within reach, at the receiver's initiative
WITHHOLD        do not surface it at all

The sidecar discipline from Part I generalises here. There it said that retrieved material should be available and not intrusive. The same applies to everything the system finds. Something can deserve to exist in the interface without deserving to interrupt, which gives a first sketch of a distinction developed later:

qualifies for attention  ≠  deserves interruption

A contradiction between the report and a runbook may be crucial to the decision she is making, and belong in the main view. A related incident from last year may deserve to be one click away. The chat excerpts may best stay out unless asked for. These are three different judgements about the same pile, and treating them as one, as a ranking from most to least relevant with a cut-off, throws away what matters most. A ranking says how much something matters. It does not say what kind of appearance it should make.

What withholding costs

There is a cost in this picture that the reader never sees. If the system shows too much, she notices at once: the interface is cluttered, her time is wasted. If it withholds the wrong thing, she notices nothing at all. She cannot miss what she never saw. The contradiction that was relevant and was dropped is invisible to her by construction.

That asymmetry should shape how selection is built. A wasted glance costs seconds. A missed item can cost a decision, and the reader has no way to know it happened. Any selection policy therefore has to be answerable for what it left out as well as for what it put in. The reader should be able to see that something was withheld and, where it matters, why, and the choice should be reversible: a click should bring back what was left out. A system that filters silently is asking to be trusted with a power it has given the reader no means of checking.

Importance is not a property of the item

A last point shapes the rest of the part. It is natural to think of selection as ranking items by how important they are, with importance as a fixed property of each item. This does not hold up. Two competent engineers handed the same incident and asked what matters most will not choose the same things, and the difference is not noise. One is about to approve a fix, one is writing a review, one is deciding whether her own service is affected. What matters is a relation between an item and a purpose, a reader and a moment. That is an argument and not a measured result, and it will be tested against cases as the part proceeds, but it already tells us what a selection system cannot be. It cannot be a single global ranking, however good. It has to take the purpose, the person and the moment into account, which brings in problems of its own about what the system may know and how it may use it.

The questions ahead

The rest of this part follows the ladder. First, selection needs something to select among, and most sources are not tidy documents: an hour of video or a long thread has to be broken into pieces before any of it can be chosen. Then there is the question of what makes a piece worth keeping in the source’s own terms, then for a particular purpose, then for a particular person who may already know it. After that come standing interests that let a reader ask to be told when something changes, the idea that information can reach someone at several levels of detail, the limit beyond which compressing it destroys what mattered, and the timing of when anything should enter the foreground. The part ends by putting these together.

Where this leaves us

A system that can generate representations has to decide which of them should reach a finite receiver. The decision has stages, and what exists is not what is relevant, what is selected, what is shown, what is noticed or what is useful. Abundance is a property of the world and overload a mismatch among supply, capacity, task, time and the cost of looking. A candidate can be shown now, made available or withheld, and qualifying for attention is not deserving interruption. Withholding has a cost the receiver cannot see, so selection must stay accountable for what it leaves out. And importance is a relation among an item, a purpose and a person, not a number attached to the item.

Not everything useful deserves to be shown now. Before a system can decide which things do, it needs to know what the things are. That is easy to say for a paper and surprisingly hard for an hour of video, which is where the next chapter begins.

References

• Simon, H.A. (1971). Designing Organizations for an Information-Rich World. In M. Greenberger (ed.), Computers, Communications, and the Public Interest. Baltimore: Johns Hopkins Press, pp. 37–72.