Validate the whole proposal before any side effect
Run a proposal through the action boundary and see the schema, semantic and authorization failures kept apart, with the executor only ever receiving a typed action.
A model can propose more than a runtime should do, and the chapter’s claim is that the boundary has to answer three separate questions in order, and fail loudly at each. Does the proposal have the right shape? Are the values usable? And may this run perform it? A proposal that passes the first two can still be refused by the third.
Run it
The boundary is the chapter’s own code. Save the first block as
action_boundary.py, save the second as demo.py beside it, and run the
driver with Python 3.11 or later. The standard library only: no model, no
network, no service.
Expected outcome
a well-formed search
accepted: SearchAction -> ['result 0 for retrieval contract', 'result 1 for retrieval contract', 'result 2 for retrieval contract']
an unknown action
refused: SchemaError: Unknown action: publish
an invented extra argument
refused: SchemaError: Unexpected fields in search arguments: ['depth']
a missing argument
refused: SchemaError: Missing fields in search arguments: ['max_results']
a well-formed but empty query
refused: SemanticError: query cannot be empty
a well-formed but out-of-range count
refused: SemanticError: max_results must be between 1 and 20
a well-formed final answer, not allowed in this run
refused: PermissionError: Action not allowed in this run: final
tool calls made: ["search('retrieval contract', max_results=3)"]
the executor only ever received a typed action: TrueOne tool call was made, for the one proposal that cleared every rung. The
final action is well formed: it has the right keys and usable values, and
it is still refused, because the run’s policy does not permit it.
The boundary
The chapter’s code, in full, with two validators added in the shape the
chapter specifies for them. The action types, the two exception types, the
exact-key check, the search validator, the authorisation step and the executor
are the chapter’s own; validate_calculate and validate_final are written
here in the shape the chapter describes for them.
"""The action boundary from the agents-from-first-principles chapter 2."""
from dataclasses import dataclass
from enum import StrEnum
from typing import Literal
class Stage(StrEnum):
REPRESENTATION = "representation"
SCHEMA = "schema"
SEMANTICS = "semantics"
AUTHORIZATION = "authorization"
PRECONDITION = "precondition"
EXECUTION = "execution"
@dataclass(frozen=True)
class SearchAction:
kind: Literal["search"]
query: str
max_results: int
@dataclass(frozen=True)
class CalculateAction:
kind: Literal["calculate"]
expression: str
@dataclass(frozen=True)
class FinalAction:
kind: Literal["final"]
answer: str
Action = SearchAction | CalculateAction | FinalAction
class SchemaError(ValueError):
"""Structure, keys or primitive types are wrong."""
class SemanticError(ValueError):
"""Structure is correct; the values are not usable."""
def require_exact_keys(data: dict, expected: set[str], *, where: str) -> None:
actual = set(data)
missing = expected - actual
extra = actual - expected
if missing:
raise SchemaError(f"Missing fields in {where}: {sorted(missing)}")
if extra:
raise SchemaError(f"Unexpected fields in {where}: {sorted(extra)}")
def validate_search(args: dict) -> SearchAction:
require_exact_keys(args, {"query", "max_results"}, where="search arguments")
query, max_results = args["query"], args["max_results"]
if not isinstance(query, str):
raise SchemaError("query must be a string")
if type(max_results) is not int:
raise SchemaError("max_results must be an integer")
query = query.strip()
if not query:
raise SemanticError("query cannot be empty")
if not 1 <= max_results <= 20:
raise SemanticError("max_results must be between 1 and 20")
return SearchAction("search", query, max_results)
def validate_calculate(args: dict) -> CalculateAction:
require_exact_keys(args, {"expression"}, where="calculate arguments")
expression = args["expression"]
if not isinstance(expression, str):
raise SchemaError("expression must be a string")
expression = expression.strip()
if not expression:
raise SemanticError("expression cannot be empty")
return CalculateAction("calculate", expression)
def validate_final(args: dict) -> FinalAction:
require_exact_keys(args, {"answer"}, where="final arguments")
answer = args["answer"]
if not isinstance(answer, str):
raise SchemaError("answer must be a string")
return FinalAction("final", answer.strip())
def validate_proposal(data: object) -> Action:
if not isinstance(data, dict):
raise SchemaError("Action proposal must be an object")
require_exact_keys(data, {"action", "arguments"}, where="action proposal")
name, arguments = data["action"], data["arguments"]
if not isinstance(name, str):
raise SchemaError("action must be a string")
if not isinstance(arguments, dict):
raise SchemaError("arguments must be an object")
validators = {
"search": validate_search,
"calculate": validate_calculate,
"final": validate_final,
}
if name not in validators:
raise SchemaError(f"Unknown action: {name}")
return validators[name](arguments)
@dataclass(frozen=True)
class RunPolicy:
allowed_actions: frozenset[str]
def authorize(action: Action, policy: RunPolicy) -> None:
if action.kind not in policy.allowed_actions:
raise PermissionError(f"Action not allowed in this run: {action.kind}")
def execute(action: Action, tools):
if isinstance(action, SearchAction):
return tools.search(query=action.query, max_results=action.max_results)
if isinstance(action, CalculateAction):
return tools.calculate(action.expression)
if isinstance(action, FinalAction):
return action.answer
raise TypeError(f"Unsupported validated action: {type(action)}")The driver
The driver is not part of the chapter. It runs seven proposals through the boundary and prints what happened to each.
"""Driver for the agents-from-first-principles chapter 2 action boundary."""
from action_boundary import (
Action,
FinalAction,
RunPolicy,
SchemaError,
SearchAction,
SemanticError,
authorize,
execute,
validate_proposal,
)
class Tools:
def __init__(self) -> None:
self.calls: list[str] = []
def search(self, *, query: str, max_results: int):
self.calls.append(f"search({query!r}, max_results={max_results})")
return [f"result {index} for {query}" for index in range(max_results)]
def calculate(self, expression: str):
self.calls.append(f"calculate({expression!r})")
return f"value of {expression}"
tools = Tools()
policy = RunPolicy(allowed_actions=frozenset({"search", "calculate"}))
def attempt(label, proposal):
try:
action = validate_proposal(proposal)
except (SchemaError, SemanticError) as error:
print(f"{label}\n refused: {type(error).__name__}: {error}")
return
try:
authorize(action, policy)
except PermissionError as error:
print(f"{label}\n refused: PermissionError: {error}")
return
result = execute(action, tools)
print(f"{label}\n accepted: {type(action).__name__} -> {result!r}")
attempt(
"a well-formed search",
{"action": "search", "arguments": {"query": "retrieval contract", "max_results": 3}},
)
attempt(
"an unknown action",
{"action": "publish", "arguments": {"answer": "done"}},
)
attempt(
"an invented extra argument",
{"action": "search", "arguments": {"query": "x", "max_results": 3, "depth": 9}},
)
attempt(
"a missing argument",
{"action": "search", "arguments": {"query": "x"}},
)
attempt(
"a well-formed but empty query",
{"action": "search", "arguments": {"query": " ", "max_results": 3}},
)
attempt(
"a well-formed but out-of-range count",
{"action": "search", "arguments": {"query": "x", "max_results": 500}},
)
attempt(
"a well-formed final answer, not allowed in this run",
{"action": "final", "arguments": {"answer": "the answer"}},
)
print()
print("tool calls made:", tools.calls)
print("the executor only ever received a typed action:", all(
isinstance(a, (SearchAction, FinalAction)) for a in [validate_proposal(
{"action": "search", "arguments": {"query": "x", "max_results": 1}})]
))Mechanism and limitations
Documented: the stage vocabulary, the typed actions, the two exception types, the exact-key rule, the search validator, the authorisation step and the executor are in the chapter’s own text, and the code above is that text’s code. The chapter also names the failure it is preparing for: a half-interpreted proposal that has already begun writing files leaves the system in a state no rung of the ladder approved.
Observed: the run above. An unknown action and an invented extra argument
were refused as schema failures; an empty query and an out-of-range count were
refused as semantic failures; a well-formed final answer was refused by
authorisation; and the executor was called once, with a SearchAction.
Not shown. The chapter’s validate_calculate and validate_final follow
the same shape as the search validator and are written here in that shape; the
chapter states the pattern rather than showing those two in full. The boundary
here stops at authorisation: the chapter’s precondition rung and its repair
policy are not in this program, so a refused proposal simply stops. The tools
are a stub. Nothing here shows that a model produces better proposals because
the boundary is sharper, and nothing here shows that a refusal is understood.
Understand this example
Copy this prompt into your AI tool. No code runs here.
Apply this example
Copy this prompt into your AI tool. No code runs here.