Browser AI From First Principles
Chapter 03 of 24

Build the Smallest Browser AI Observatory

By AI, Browser AI

Explain this chapter with AI

Copy this prompt into ChatGPT, Claude, Gemini, a local model, or another AI.

Apply this chapter with AI

Copy this prompt into ChatGPT, Claude, Gemini, a local model, or another AI.

We are ready to make the browser’s AI lifecycle visible.

The first version will be deliberately small. It will not inspect every page, intercept arbitrary prompts, call tools, diagnose quality or persist a history. It will do five things:

  1. create a panel inside Chrome DevTools;
  2. report whether the Prompt API is exposed and available;
  3. create a browser-managed language-model session with visible download progress;
  4. stream a prompt with cancellation;
  5. display an event trace that separates each phase.

That is already more useful than a chat demo.

A chat demo answers, “Can I get some text?”

Our first observatory answers:

Was the API exposed?
What capability did we request?
What availability state did Chrome report?
Did model acquisition occur?
How long did session creation take?
How long until the first output?
Did the request complete, fail or get cancelled?

This chapter builds the complete extension from plain HTML, CSS and JavaScript. No bundler is required. That keeps the browser boundary visible; a later implementation can add TypeScript and a build system after the architecture has earned them.

1. Why build a DevTools extension?

A toolbar popup disappears when it loses focus. A side panel is useful for page-level assistants, but our product is an engineering instrument. It belongs beside the inspected page’s Elements, Console, Network and Performance panels.

Chrome lets an extension add a custom DevTools panel through a devtools_page entry in manifest.json.

The resulting architecture has three contexts:

    flowchart TD
    M[Manifest V3 extension] --> D[DevTools page]
    D --> P[Observatory panel]
    D --> W[Inspected window APIs]
    P --> L[Built-in LanguageModel]

For this first slice, the DevTools page only registers the panel. The panel owns its own Prompt API session and its trace.

That last sentence matters. We are not yet claiming to observe Prompt API calls made by the inspected application. We are building an extension-owned test console located inside DevTools. Chapter 4 will design the opt-in bridge required to receive application traces without pretending DevTools exposes events it does not.

2. Create the extension skeleton

Create this directory:

browser-ai-observatory/
  manifest.json
  devtools.html
  devtools.js
  panel.html
  panel.css
  panel.js

The manifest is intentionally narrow:

{
  "manifest_version": 3,
  "name": "Browser AI Observatory",
  "version": "0.1.0",
  "description": "Inspect and experiment with browser-managed AI from DevTools.",
  "devtools_page": "devtools.html"
}

There are no host permissions, content scripts, background service worker or network permissions.

This is not merely minimalism. Permission scope is part of the product’s trust model. The extension can run its own local experiment without reading every page the user visits, so version 0.1.0 should not request that authority.

The DevTools bootstrap is a local HTML page:

<!doctype html>
<html lang="en">
  <head>
    <meta charset="utf-8">
    <script src="devtools.js" defer></script>
  </head>
  <body></body>
</html>

The JavaScript registers our panel:

chrome.devtools.panels.create(
  "AI Observatory",
  "",
  "panel.html"
);

Chrome keeps an instance of the DevTools page alive while that DevTools window remains open. The panel page is loaded when the panel is shown.

3. Build the panel as an instrument

The interface needs controls, state and a trace. It does not need a chat persona.

<!doctype html>
<html lang="en">
  <head>
    <meta charset="utf-8">
    <meta name="viewport" content="width=device-width, initial-scale=1">
    <title>Browser AI Observatory</title>
    <link rel="stylesheet" href="panel.css">
    <script type="module" src="panel.js"></script>
  </head>
  <body>
    <header>
      <div>
        <p class="eyebrow">Browser AI Observatory</p>
        <h1>Local model trace</h1>
      </div>
      <output id="status" class="status">Not inspected</output>
    </header>

    <main>
      <section class="card controls" aria-labelledby="runtime-heading">
        <h2 id="runtime-heading">Runtime</h2>
        <dl class="facts">
          <div><dt>API</dt><dd id="api-state">Unknown</dd></div>
          <div><dt>Availability</dt><dd id="availability">Unknown</dd></div>
          <div><dt>Session</dt><dd id="session-state">None</dd></div>
        </dl>

        <progress id="download" max="1" value="0" hidden></progress>

        <div class="actions">
          <button id="inspect">Inspect capability</button>
          <button id="create" disabled>Create session</button>
        </div>
      </section>

      <section class="card" aria-labelledby="prompt-heading">
        <h2 id="prompt-heading">Prompt</h2>
        <label for="prompt">Input</label>
        <textarea id="prompt" rows="5">Explain why an availability check and a session are different.</textarea>

        <div class="actions">
          <button id="run" disabled>Run</button>
          <button id="stop" disabled>Stop</button>
        </div>

        <label for="response">Streamed response</label>
        <output id="response" class="response"></output>
      </section>

      <section class="card trace-card" aria-labelledby="trace-heading">
        <div class="section-title">
          <h2 id="trace-heading">Trace</h2>
          <button id="clear" class="secondary">Clear</button>
        </div>
        <ol id="trace" class="trace"></ol>
      </section>
    </main>
  </body>
</html>

The layout uses semantic elements so that the extension remains usable with keyboard and assistive technology. A debugger is not exempt from product quality because its users are developers.

The styling can stay compact:

:root {
  color-scheme: light dark;
  font: 13px/1.45 system-ui, sans-serif;
  --border: color-mix(in srgb, CanvasText 18%, transparent);
  --muted: color-mix(in srgb, CanvasText 62%, transparent);
  --accent: #5b7cfa;
}

* { box-sizing: border-box; }

body {
  margin: 0;
  color: CanvasText;
  background: Canvas;
}

header, main { padding: 16px; }

header {
  display: flex;
  align-items: start;
  justify-content: space-between;
  border-bottom: 1px solid var(--border);
}

h1, h2, p { margin-top: 0; }
h1 { margin-bottom: 0; font-size: 18px; }
h2 { font-size: 14px; }
.eyebrow { margin-bottom: 2px; color: var(--muted); }

main {
  display: grid;
  grid-template-columns: minmax(280px, 0.8fr) minmax(360px, 1.2fr);
  gap: 12px;
}

.card {
  padding: 14px;
  border: 1px solid var(--border);
  border-radius: 8px;
}

.trace-card { grid-column: 1 / -1; }

.facts {
  display: grid;
  grid-template-columns: repeat(3, 1fr);
  gap: 8px;
}

.facts div { padding: 8px; background: color-mix(in srgb, CanvasText 5%, transparent); }
dt { color: var(--muted); }
dd { margin: 2px 0 0; font-weight: 600; }

textarea, .response, progress { width: 100%; }
textarea { resize: vertical; }

.response {
  display: block;
  min-height: 120px;
  margin-top: 6px;
  padding: 10px;
  white-space: pre-wrap;
  border: 1px solid var(--border);
}

.actions, .section-title {
  display: flex;
  gap: 8px;
  align-items: center;
  margin: 10px 0;
}

.section-title { justify-content: space-between; }
button { padding: 6px 10px; }
.status { color: var(--muted); }

.trace {
  max-height: 280px;
  overflow: auto;
  padding-left: 28px;
  font-family: ui-monospace, monospace;
}

.trace li { padding: 3px 0; }
.trace time { color: var(--muted); margin-right: 8px; }

The CSS uses system colors and color-scheme, so the panel follows the DevTools theme without needing theme detection in the first version.

4. Treat events as the primary output

Before adding model calls, define the trace.

const traceElement = document.querySelector("#trace");
const events = [];

function now() {
  return performance.timeOrigin + performance.now();
}

function emit(type, detail = {}) {
  const event = Object.freeze({
    id: crypto.randomUUID(),
    type,
    at: now(),
    detail,
  });

  events.push(event);
  renderEvent(event);
  return event;
}

function renderEvent(event) {
  const item = document.createElement("li");
  const time = document.createElement("time");
  const message = document.createElement("span");

  time.dateTime = new Date(event.at).toISOString();
  time.textContent = new Date(event.at).toLocaleTimeString();
  message.textContent = `${event.type} ${JSON.stringify(event.detail)}`;

  item.append(time, message);
  traceElement.append(item);
  item.scrollIntoView({ block: "nearest" });
}

Using textContent rather than HTML insertion is a security property. Error messages, model output and future page-provided fields are data. They are not trusted markup.

The trace is an append-only sequence for the life of the panel. UI facts such as “Available” are projections of that sequence. Later we can persist, filter and replay it without redesigning the fundamental record.

5. Run the Observatory inside this chapter

The DevTools extension is the product we are building. The chapter experience is its immediately runnable counterpart.

Select Run with Browser AI from this chapter to open:

/tools/ai/browser-ai-from-first-principles/03-chapter/

The page performs the same core sequence as the extension:

inspect capability
      ↓
create session
      ↓
run streaming prompt
      ↓
append versioned events
      ↓
derive measurements
      ↓
export trace

There is one deliberate difference. The chapter experience runs as a top-level web page, while the extension runs in a DevTools panel. Each instrument observes only the Prompt API work it owns. Neither can infer that it has intercepted private model calls made elsewhere merely because it displays an event console.

The Observatory summary is a projection of the event stream, not a second source of timing data. Five values are derived from recorded events:

Measurement Source event
Event count Length of the current event sequence
Session creation session.create.finished.data.durationMs
First chunk prompt.finished.data.timeToFirstChunkMs
Total prompt prompt.finished.data.totalMs
Outcome Latest terminal prompt or session event

This matters because duplicated measurement logic drifts. If one timer feeds the dashboard and another timer feeds the export, the screen and the evidence can disagree. By deriving the screen from events, replay and live execution use the same calculation path.

Missing evidence remains visible as missing. The reviewed September 2 trace contains ten events and a completed session creation, but no completed prompt. Its summary therefore shows the session duration while first-chunk time and total prompt time remain —. The interface must not estimate those values or borrow them from another run merely to fill the cards.

The chapter-specific bundle keeps executable behavior separate from explanation and evidence:

03-chapter/context.md          concepts supplied to the model
03-chapter/experiment.json     prompt, options and requested view
03-chapter/recorded-trace.json reviewed metrics-only evidence

The Prompt API options include samplingMode: "most-predictable". This makes the experiment compatible with the speculative-decoding constraint we encountered in the real Chrome build. The same options are supplied to availability inspection and session creation.

The actual Manifest V3 implementation remains under:

experiments/browser-ai-from-first-principles/browser-ai-observatory/

The site experience does not replace that extension. It gives every reader an executable version of the chapter’s central architecture, provides a replay path when live Browser AI is unavailable, and establishes the dashboard contract we can reuse as the Observatory grows.

6. Inspect the capability

Define the session options once:

const sessionOptions = Object.freeze({
  samplingMode: "most-predictable",
  expectedInputs: [{ type: "text", languages: ["en"] }],
  expectedOutputs: [{ type: "text", languages: ["en"] }],
});

Then connect the interface:

const ui = {
  status: document.querySelector("#status"),
  apiState: document.querySelector("#api-state"),
  availability: document.querySelector("#availability"),
  sessionState: document.querySelector("#session-state"),
  download: document.querySelector("#download"),
  inspect: document.querySelector("#inspect"),
  create: document.querySelector("#create"),
  prompt: document.querySelector("#prompt"),
  run: document.querySelector("#run"),
  stop: document.querySelector("#stop"),
  response: document.querySelector("#response"),
  clear: document.querySelector("#clear"),
};

let session = null;
let sessionId = null;
let activeController = null;

ui.inspect.addEventListener("click", inspectCapability);

async function inspectCapability() {
  emit("capability.inspect.started", { api: "LanguageModel" });

  if (!("LanguageModel" in globalThis)) {
    ui.apiState.textContent = "Not exposed";
    ui.availability.textContent = "Unavailable";
    ui.status.textContent = "Prompt API is not exposed in this context";
    ui.create.disabled = true;
    emit("capability.inspect.finished", {
      exposed: false,
      availability: "unavailable",
    });
    return;
  }

  ui.apiState.textContent = "Exposed";

  try {
    const availability = await LanguageModel.availability(sessionOptions);
    ui.availability.textContent = availability;
    ui.create.disabled = availability === "unavailable";
    ui.status.textContent = availabilityMessage(availability);
    emit("capability.inspect.finished", { exposed: true, availability });
  } catch (error) {
    ui.status.textContent = `Inspection failed: ${error.message}`;
    emit("capability.inspect.failed", serializeError(error));
  }
}

function availabilityMessage(value) {
  return {
    available: "Ready to create a session",
    downloadable: "A model download is required",
    downloading: "The model is downloading",
    unavailable: "This capability is unavailable",
  }[value] ?? `Unknown availability: ${value}`;
}

function serializeError(error) {
  return {
    name: error?.name ?? "Error",
    message: error?.message ?? String(error),
  };
}

We store the error name and message, not the entire error object. Error objects do not serialize consistently across extension boundaries, and stack traces can contain local or extension-specific information we may not want to persist by default.

7. Create a session and monitor acquisition

Session creation belongs to its own user action.

ui.create.addEventListener("click", createSession);

async function createSession() {
  ui.create.disabled = true;
  ui.inspect.disabled = true;
  ui.sessionState.textContent = "Creating";
  ui.status.textContent = "Creating local session";

  const startedAt = performance.now();
  emit("session.create.started", { options: sessionOptions });

  try {
    session = await LanguageModel.create({
      ...sessionOptions,
      monitor(monitor) {
        ui.download.hidden = false;
        monitor.addEventListener("downloadprogress", (event) => {
          ui.download.value = event.loaded;
          ui.status.textContent = `Downloading ${Math.round(event.loaded * 100)}%`;
          emit("model.download.progress", { loaded: event.loaded });
        });
      },
    });

    sessionId = crypto.randomUUID();
    ui.download.hidden = true;
    ui.sessionState.textContent = "Ready";
    ui.status.textContent = "Session ready";
    ui.run.disabled = false;

    emit("session.create.finished", {
      sessionId,
      durationMs: performance.now() - startedAt,
    });
  } catch (error) {
    session = null;
    sessionId = null;
    ui.sessionState.textContent = "Failed";
    ui.status.textContent = `Session failed: ${error.message}`;
    ui.create.disabled = false;
    emit("session.create.failed", {
      durationMs: performance.now() - startedAt,
      error: serializeError(error),
    });
  } finally {
    ui.inspect.disabled = false;
  }
}

The monitor may not emit progress on a warm machine because no model acquisition is needed. The event trace therefore treats download as an optional phase rather than assuming every creation passes through it.

We also measure session creation independently of download. A warm session still has a creation cost, and the time after download reaches 1 can reveal preparation work that a progress bar alone conceals.

8. Stream, measure and stop

The prompt runner creates a new trace identity for every request.

ui.run.addEventListener("click", runPrompt);
ui.stop.addEventListener("click", () => activeController?.abort());

async function runPrompt() {
  const prompt = ui.prompt.value.trim();
  if (!session || !prompt) return;

  const traceId = crypto.randomUUID();
  const startedAt = performance.now();
  let firstChunkAt = null;
  let chunks = 0;
  let outputChars = 0;

  activeController = new AbortController();
  ui.run.disabled = true;
  ui.stop.disabled = false;
  ui.response.textContent = "";
  ui.status.textContent = "Generating";

  emit("prompt.started", {
    traceId,
    sessionId,
    inputChars: [...prompt].length,
    capture: "metrics-only",
  });

  try {
    const stream = session.promptStreaming(prompt, {
      signal: activeController.signal,
    });

    for await (const chunk of stream) {
      const chunkAt = performance.now();
      firstChunkAt ??= chunkAt;
      chunks += 1;
      outputChars += [...chunk].length;
      ui.response.append(document.createTextNode(chunk));

      emit("prompt.chunk", {
        traceId,
        index: chunks,
        chars: [...chunk].length,
        elapsedMs: chunkAt - startedAt,
      });
    }

    emit("prompt.finished", {
      traceId,
      outcome: "completed",
      chunks,
      outputChars,
      timeToFirstChunkMs:
        firstChunkAt === null ? null : firstChunkAt - startedAt,
      totalMs: performance.now() - startedAt,
    });
    ui.status.textContent = "Complete";
  } catch (error) {
    const outcome = error.name === "AbortError" ? "aborted" : "failed";
    emit("prompt.finished", {
      traceId,
      outcome,
      chunks,
      outputChars,
      totalMs: performance.now() - startedAt,
      error: outcome === "failed" ? serializeError(error) : null,
    });
    ui.status.textContent = outcome === "aborted" ? "Stopped" : `Failed: ${error.message}`;
  } finally {
    activeController = null;
    ui.run.disabled = false;
    ui.stop.disabled = true;
  }
}

Prompt content is visible in the input because the user typed it. It is not copied into the trace. The event records capture: "metrics-only" and an input size.

Later we can add an explicit content-capture mode for developers debugging their own application. It must be an opt-in setting, visible in the panel, scoped to the inspected origin, and disabled again automatically according to a clear policy.

The clear button only resets the in-memory view:

ui.clear.addEventListener("click", () => {
  events.length = 0;
  traceElement.replaceChildren();
  emit("trace.cleared");
});

At this stage there is no hidden persistent copy.

9. Load and test the extension

Use a Chrome build that supports the Prompt API configuration you are testing. Experimental channels and flags change over time, so treat the current Chrome documentation and your EPP instructions as the source of truth rather than hard-coding a version claim into the extension.

Then:

  1. Open chrome://extensions.
  2. Enable Developer mode.
  3. Choose Load unpacked.
  4. Select the browser-ai-observatory directory.
  5. Open a normal web page.
  6. Open DevTools.
  7. Select AI Observatory.
  8. Choose Inspect capability.

Test the lifecycle, not just the answer.

Test Expected observation
API not exposed Panel reports structural absence without throwing
Capability unavailable Create remains disabled
Cold model Download events appear before session completion
Warm model Session can be created without assuming progress events
Normal prompt First-chunk and total timing are recorded
Stop during generation Outcome is aborted, not failed
Empty prompt No request is made
Clear trace Existing rows disappear and a new clear event begins the sequence

If the API should be present but is not functioning, Chrome exposes chrome://on-device-internals for model status and diagnostics. Our panel complements that browser-internal view. It does not replace it.

10. What this first version proves

The extension proves that a browser-managed model can participate in an ordinary extension interface without an application server or model API key.

More importantly, it reveals the lifecycle hidden by a one-line prompt demo.

The experiment should produce a trace shaped like this:

capability.inspect.started
capability.inspect.finished { availability: "downloadable" }
session.create.started
model.download.progress { loaded: 0.04 }
...
model.download.progress { loaded: 1 }
session.create.finished { durationMs: ... }
prompt.started { capture: "metrics-only" }
prompt.chunk { index: 1, elapsedMs: ... }
...
prompt.finished { outcome: "completed", totalMs: ... }

That trace can be compared, stored, summarized and eventually evaluated. A console transcript cannot reliably do those jobs.

11. What it does not prove

It does not prove that the output is correct.

It does not prove that local inference is faster than a remote alternative.

It does not prove that content never leaves the machine; we have only shown that this model call does not require our own model server.

It does not observe calls made privately inside the inspected page.

It does not identify the precise model version unless the browser exposes that fact through a supported interface.

It does not survive panel closure.

It does not yet correlate a model response with a page event, network request, WebMCP tool or product outcome.

Those limits are not defects in the chapter. They are the requirements for the next one.

Conclusion

We have built the smallest useful Browser AI Observatory.

It is a real DevTools extension, but its most important component is not the panel. It is the event model. The panel now exposes capability inspection, optional model acquisition, session creation, streaming generation, time to first output, total duration, cancellation and operational failure as distinct events.

The chapter now runs that model as well as explaining it. Its live page derives the dashboard from the same events it exports, while replay mode demonstrates that absent measurements remain absent. The result is our first chapter whose prose, executable experience and extension source describe the same instrument from three directions.

The implementation also preserves an honest boundary: it observes work the extension owns. It does not claim magical access to AI calls inside an inspected application.

The next chapter crosses that boundary deliberately. We will design an opt-in instrumentation bridge, separate page data from extension control, define a versioned trace envelope, and turn the experiment console into the beginning of a genuine monitor and debugger.

Sources and further reading

  1. Chrome for Developers, Extend DevTools.
  2. Chrome for Developers, chrome.devtools.panels.
  3. Chrome for Developers, The Prompt API.
  4. Chrome for Developers, Inform users of model download.
  5. Chrome for Developers, Get started with built-in AI.