Refuse a tool call that cannot be executed
Run a mocked calculator loop, count its retries on an unknown tool, and replace its unsafe eval with the repository's AST-only evaluator.
A tool call is a request to execute code with model-supplied arguments. The
serving layer must decide whether the tool exists, whether the arguments are
valid, what the tool may touch, and how to report a refusal. A loop that catches
every exception and retries forever turns a bad call into latency; a loop that
passes the argument to eval turns it into a security incident.
This example runs a mocked calculator loop with two valid expressions and one
unknown tool. The valid calls return 4 and 42; the unknown tool returns
error after three attempts. The implementation uses Python’s eval with
builtins removed, which is not a safe sandbox. The repository’s
tool_safe.py is the alternative: it parses an AST and permits only numbers and
six arithmetic operators, with length, node, integer and exponent bounds.
Run it
From the canonical repository at commit
f958967d70992d7b09d9c75000bd8c543f1644d4:
python examples/ch34/tool_loop.py --jsonPython 3.11 or later, standard library only. No model, network, GPU or credentials. It finishes immediately. The trace ids are random UUIDs and are not quoted; the tool, arguments, result and retry count are stable.
Implementation
call_tool retries while the budget remains, returns error when it is
exhausted, and wraps every result in a trace id. The loop is a mock; the
important boundary is that an unknown tool is refused rather than treated as a
successful empty result.
class ToolLoop:
def __init__(self, max_retries: int = 2) -> None:
if max_retries < 0:
raise ValueError("max_retries must be non-negative")
self.max_retries = max_retries
def call_tool(self, tool: str, args: str) -> ToolResult:
trace_id = str(uuid.uuid4())
retries = 0
result = None
while retries <= self.max_retries:
try:
if tool == "calc":
result = str(eval(args, {"__builtins__": {}}, {}))
else:
raise ValueError("unknown tool")
break
except Exception:
retries += 1
if retries > self.max_retries:
result = "error"
break
return ToolResult(trace_id=trace_id, tool=tool, args=args, result=result, retries=retries)Expected outcome
Observed with the command above (exit 0). Selected stable fields:
"tool": "calc",
"args": "2+2",
"result": "4",
"retries": 0,
"tool": "unknown",
"args": "x",
"result": "error",
"retries": 3,The first two calls succeeded. The unknown tool was attempted three times
(max_retries=2 means the initial attempt plus two retries) and then returned
error. The trace ids in the full output are UUIDs, so they are omitted rather
than pasted as if they were stable evidence.
Mechanism and limitations
Documented: ToolLoop rejects a negative retry count, wraps every call in a
trace id, retries until the budget is exhausted, and returns error on failure.
The test file checks the calculator results, the trace id, the unknown tool, the
demo, JSON serialisation and the invalid retry configuration. tool_safe.py
documents the safe alternative and its bounds.
Observed: the command exited 0 with the results and retry count above. The
targeted tests.test_ch34_tool is part of the canonical suite; no independent
test count is claimed here.
Not shown: a real model choosing tools, tool-call protocol messages, argument
schema validation, process isolation, or a security proof. eval with
__builtins__ removed is still not a safe evaluator; the page does not recommend
it. tool_safe.py runs in a child process for crash isolation, but its own
docstring says that is not a security boundary against an evaluator hole. No
model or hosted tool API was called.
Challenge this result
Run "__import__('os').system('echo unsafe')" through the mock’s calculator and
explain why removing builtins does not make it safe. Then run the same expression
through tool_safe.safe_eval and inspect its refusal code. Add an argument schema
that accepts only two integer operands and refuses a string, and state where the
refusal should happen: in the model’s prompt, in the tool schema, or in the
executor. Only one of those is a security boundary.
Source and verification
Canonical source:
examples/ch34/tool_loop.py
and the safe evaluator
examples/ch34/tool_safe.py
at commit f958967d70992d7b09d9c75000bd8c543f1644d4. Tests:
tests/test_ch34_tool.py
.
Chapter: Tool Calling and Agentic Inference
.
The run above is a fresh local execution on Windows 11 with Python 3.13.15; it
uses a mocked tool loop, not a model or a hosted tool service.
Understand this example
Copy this prompt into your AI tool. No code runs here.
Apply this example
Copy this prompt into your AI tool. No code runs here.