Browser AI From First Principles

For most of the history of web applications, the browser has occupied a clear position in the architecture.

It displays an interface.

It executes application code.

It stores local state.

It communicates with servers that perform the expensive work.

AI initially fitted neatly into that arrangement. A web application collected context, sent it across the network to a model running in a data centre, received a response, and displayed the result.

That architecture is changing.

Chrome’s built-in AI work allows pages and extensions to request AI capabilities backed by models managed by the browser. At the same time, WebMCP gives websites a way to expose structured tools to browser agents.

Put those developments together:

model inside browser
        ↓
context inside browser
        ↓
tools inside browser
        ↓
agent inside browser

This book studies that transition from first principles.

The durable question is not which model happens to ship in one version of Chrome. It is:

What happens when the browser stops being merely an interface to AI and becomes an AI runtime itself?

The book is also a tool

This project goes beyond explaining APIs.

Throughout the book we will build one system through two reader-facing stages.

Stage One is the Browser AI Laboratory: a full-page experience in which every chapter becomes a live experiment with capability detection, real execution when supported, captured-trace replay when it is not, and an inspectable evidence trail.

Stage Two is the AI-enabled site: the books and wider website become structured, retrievable context with typed operations, interchangeable AI runtimes and policies controlled by the user.

Browser AI Observatory is the evidence layer underneath both stages. Its DevTools extension begins as a capability probe and streaming prompt console, then grows into an application monitor, trace viewer, profiler, evaluation harness, WebMCP agent inspector and personal browser policy engine.

The extension will help answer questions that a normal chat interface hides:

• Is the API absent, unsupported, downloading, loading or ready?
• How long did session creation take?
• How long did the user wait for the first output?
• Which prompt used which session and capability options?
• Did a request complete, fail or get cancelled?
• Did successful inference produce an application result that passed validation?
• Did a browser or model update change behavior on a known fixture?
• Which WebMCP tools were exposed to an agent?
• Which tool was selected, with what arguments and authority?
• Did untrusted page content influence a privileged action?
• Which rule changed a page, and what evidence justified the change?
• Which model or provider handled the content, and did any data leave the device?

The product is not saved for the final chapter. Chapter 1 establishes the laboratory shell. Every later chapter adds a runnable mechanism, while the Observatory records the evidence required to trust it. The canonical implementation lives under experiments/browser-ai-from-first-principles/ in the book repository.

    flowchart TD
    R[Research question] --> E[Experiment]
    E --> T[Observatory trace]
    T --> W[Book chapter]
    W --> F[Extension feature]
    F --> R
  

The book becomes the design record for the software.

The software becomes the experimental instrument for the book.

The method

We will add one mechanism at a time.

capability detection
        ↓
model acquisition
        ↓
session and streaming
        ↓
observability
        ↓
evaluation
        ↓
structured tools
        ↓
permissions and policy
        ↓
agent runtime

At every step we will ask:

1. What new capability did the mechanism add?
2. Which failure can it introduce?
3. What evidence tells us whether it worked?
4. What authority does it require?
5. What should the debugger record?

The aim is not to memorize experimental flags or one release’s model name. Those will change. The aim is to understand the architecture that remains when the implementation underneath the API changes.

Book structure

Part I — The Browser Becomes a Runtime

1. The Browser Stops Being Just a Client — Separate inference architectures, then define the chapter-driven laboratory and AI-enabled site we will build.
2. What Does Built-In Actually Mean? — Turn capability availability, download, session creation and prompting into an explicit state machine.
3. Build the Smallest Browser AI Observatory — Create a permission-minimal Manifest V3 DevTools extension that runs and measures a local prompt.
4. From Prompt Demo to AI Debugger — Add opt-in application instrumentation, a secure bridge, trace identity, redaction and retention.

Part II — The Built-in AI Primitives

5. One Runtime, Several Interfaces — Compare a general Prompt API with task-specific APIs without assuming one physical model.
6. Summarization Is a Contract — Test length, type, format, grounding and long-input strategies.
7. Writing, Rewriting and Proofreading Are Different Operations — Separate generation, transformation and correction by authority over the text.
8. Language Detection Is a Decision, Not an Answer — Compose ranked detection with translation, thresholds and abstention.

Part III — Sessions, Context and Local Resources

9. A Session Is Not a Stateless Function — Inspect conversation state, context consumption, cloning and destruction.
10. When the Context Window Fills — Detect overflow, compact history and preserve the right state.
11. Cold Starts, Warm Runs and Real Latency — Measure download, load, first output and throughput separately.
12. The Browser Manages the Model — Examine eligibility, updates, purging, storage pressure and implementation change.

Part IV — Reliable Browser AI

13. A Resolved Promise Is Not a Correct Answer — Build behavioral fixtures beside operational traces.
14. Evaluate the Feature, Not the Demo — Create repeatable evals, baselines and release gates.
15. When Local Is Not Available — Design explicit degradation and hybrid fallback without hiding data movement.
16. Structured Output Is Still Model Output — Parse, validate, repair and reject machine-consumed responses.

Part V — Tools and the Agentic Web

17. From Buttons to Capabilities — Understand why structured tools differ from UI actuation.
18. Expose the First WebMCP Tool — Register a typed, read-only operation and inspect its contract.
19. Tool Choice Is a Behavioral Problem — Evaluate discovery, selection, argument construction and results.
20. Untrusted Text Meets Executable Authority — Defend against malicious manifests, contaminated outputs and indirect prompt injection.

Part VI — From Agent Inspector to User-Owned Browser

21. Put a Human at the Authority Boundary — Add permission scopes, confirmations, audit trails and safe defaults.
22. The Browser as a Personal Policy Engine — Combine interchangeable AI runtimes, content units, typed evidence and declarative user rules.
23. Build an AI-Origin Content Filter — Prefer provenance, preserve uncertainty and apply reversible presentation choices.
24. The User, Not the Platform, Controls the Interface — Assemble the coordinator, workers, tools, authority boundary and policy layer into the finished system.

This outline is a working architecture. Experiments may force chapters to split, merge or move. If the evidence changes the plan, the plan should change.

What we will build

Browser AI Observatory will grow through the same parts.

The extension will maintain an explicit evidence boundary.

It can directly observe operations it owns. It can receive application events through an opt-in SDK. It can inspect browser surfaces exposed by supported DevTools APIs. It will not pretend that application-reported events are browser-attested or that private browser internals are public instrumentation.

That limitation makes the tool more credible, not less.

Experimental status

Browser AI APIs and WebMCP are evolving. Some capabilities may be stable, some may require an origin trial, and others may be restricted to experimental Chrome channels or the Built-in AI Early Preview Program at the time a chapter is written.

Every implementation chapter will therefore distinguish:

• the architectural contract;
• the currently documented public API;
• experimental configuration used for a particular run;
• observations from our own environment;
• assumptions that still need to be tested.

The book will not silently turn an EPP announcement into a universal availability claim.

The experimental build is valuable precisely because it lets us test whether an application survives changes behind the same capability boundary. It is evidence, not permanence.

The larger idea

A book normally explains software.

This project uses writing, implementation and experimentation as one loop:

research
   ↓
experiment
   ↓
chapter
   ↓
software
   ↓
real use
   ↓
new evidence
   └──────────→ research

Eventually the same browser-native AI mechanisms described here can run inside technical books and documentation: explain a selection, challenge a claim, expose prerequisites, generate an experiment, navigate related concepts and offer structured operations to an agent.

The immediate product is a live browser laboratory backed by an AI monitor and debugger. It grows into an AI-enabled site and personal browser policy engine.

The larger goal is to discover what happens when written knowledge becomes operational software.