Browser Agents on Programmer.ie: Modern AI programming

From Buttons to Capabilities

Wed, 02 Sep 2026 17:00:00 +0100

A browser agent can operate software by looking at a page, guessing which control matters and simulating a click.

That is sometimes necessary. It should not be the preferred contract between two pieces of software.

The alternative is a capability: an operation the application exposes deliberately, with a name, description, input schema, result and authority boundary.

1. A button is evidence, not an interface contract

The text “Book” may represent a purchase, reservation, navigation link or decorative heading. Position and colour help a human infer meaning, but none provides a stable machine contract.

Expose the First WebMCP Tool

Wed, 02 Sep 2026 17:15:00 +0100

Our first WebMCP tool should not send a message, edit a document or buy anything.

It should read from the book we control.

We will expose search_book: a bounded operation that finds relevant chapters and passages.

1. Start with a narrow contract

Conceptually, the page registers:

navigator.modelContext.registerTool({
  name: "search_book",
  description: "Search this book for passages related to a query.",
  inputSchema: {
    type: "object",
    properties: {
      query: { type: "string", minLength: 2, maxLength: 300 },
      limit: { type: "integer", minimum: 1, maximum: 10 }
    },
    required: ["query"],
    additionalProperties: false
  },
  execute: ({ query, limit = 5 }) => searchIndex(query, limit)
});

WebMCP is evolving, so the exact registration surface must be checked against the browser version used for an experiment. The durable architecture is name, description, schema and implementation.

Tool Choice Is a Behavioral Problem

Wed, 02 Sep 2026 17:30:00 +0100

A tool can be perfectly implemented and never used correctly.

Once an agent sees several capabilities, success depends on a behavioral pipeline:

discover → select → construct arguments → invoke → interpret result

A single “task passed” score cannot tell us which stage failed.

1. Discovery precedes choice

If the tool never enters the agent’s effective context, it cannot be selected. Record which registrations were available, filtered, truncated or rejected.

Untrusted Text Meets Executable Authority

Wed, 02 Sep 2026 17:45:00 +0100

Before tools, malicious page text could mislead a model’s answer.

After tools, the same text may influence an action.

Prompt injection becomes an authority problem when untrusted content shares a reasoning context with executable capabilities.

1. Identify the trust domains

A browser agent may combine:

user instructions;
developer policy;
page content;
tool descriptions;
tool results;
prior session state.

They are all text to the model. They are not equally authoritative.

Put a Human at the Authority Boundary

Wed, 02 Sep 2026 18:00:00 +0100

“Ask the user before doing anything dangerous” sounds like a safety design.

It is only a slogan until we define what the user sees, when approval occurs and exactly what the approval authorizes.

A useful authority boundary turns a model’s proposal into a concrete, validated effect. The user approves that effect—not a vague intention and not the model in general.

1. Classify effects before requesting permission

Not every tool call deserves the same interruption. Begin with effect classes:

The User, Not the Platform, Controls the Interface

Wed, 02 Sep 2026 18:45:00 +0100

A platform decides what enters a feed, how it is ranked, which controls are prominent and which forms of automation are available.

The user may select among the settings the platform offers. That is not the same as controlling the interface.

Browser AI creates the possibility of another layer: software acting locally, under rules the user can inspect, across sites and independently of one model provider.

This chapter assembles the system we have built and defines the limits of that claim.