Read the search rule and the promotion rule as code

The appendix is where the book’s claims meet real production modules, and its rule is that the mechanics around a model call are deterministic software that can be exercised without a model at all. These two fixtures are the mechanics: how a search branch is scored, and when a new candidate is allowed to replace the champion.

Run it

The selection rule is the chapter’s own code. Save the first block as uct.py, save the second as demo.py beside it, and run the driver with Python 3.11 or later. The standard library only: no model, no DSPy import, no network, no service.

Expected outcome

selection order and values:
  unvisited        visits=0   reward=0.0   uct=inf
  under-explored   visits=3   reward=0.9   uct=1.6038
  leader           visits=10  reward=7.0   uct=1.4141

the champion rule: promote only on a non-regressing improvement
  q_cbr=0.7        tie         -> hold
  q_cbr=0.7000005  +0.0000005  -> hold
  q_cbr=0.7005     +0.0005     -> promote, but below the practical margin
  q_cbr=0.78       +0.08       -> promote
  q_cbr=0.61       -0.09       -> hold

final champion quality: 0.78

The under-explored child scores 1.60 against the leader’s 1.41 despite a lower average reward, 0.30 against 0.70. That is the exploration term doing work rather than decoration, and it is why the selection order is unvisited, then under-explored, then leader.

The champion table is the other half. The epsilon 1e-6 stops a tie and a fifth-decimal difference from promoting; a real margin of 0.01 is what stops noise from promoting. The middle row is the interesting one: it clears the epsilon and is still flagged.

The selection rule

The chapter’s code, in full.

The selection rule is the chapter’s own code, with the import it needs to run as a file. The production form of the same method is in the reference implementation, stephanie/agents/dspy/mcts_reasoning.py.

import math


def uct_value(self, parent_visits, ucb_weight):
    if self.visits == 0:
        return float("inf")
    exploitation = self.reward / max(1, self.visits)
    exploration = ucb_weight * math.sqrt(
        max(1e-9, math.log(max(1, parent_visits)) / self.visits)
    )
    return exploitation + exploration

The driver

The driver is not part of the chapter. It builds the three-child table the chapter names, and then applies the promotion rule to the five quality pairs the chapter names.

"""Driver for the dspy-from-first-principles chapter 21 fixtures."""

from uct import uct_value


class Node:
    def __init__(self, name, visits, reward):
        self.name = name
        self.visits = visits
        self.reward = reward

    def uct(self, parent_visits, ucb_weight):
        return uct_value(self, parent_visits, ucb_weight)


PARENT_VISITS = 13
UCB_WEIGHT = 1.41

children = [
    Node("unvisited", 0, 0.0),
    Node("under-explored", 3, 0.9),
    Node("leader", 10, 7.0),
]

print("selection order and values:")
for child in sorted(children, key=lambda c: c.uct(PARENT_VISITS, UCB_WEIGHT), reverse=True):
    value = child.uct(PARENT_VISITS, UCB_WEIGHT)
    shown = "inf" if value == float("inf") else f"{value:.4f}"
    print(f"  {child.name:<16} visits={child.visits:<3} reward={child.reward:<5} uct={shown}")

print()
print("the champion rule: promote only on a non-regressing improvement")

AB_DELTA_EPS = 1e-6
PRACTICAL_MARGIN = 0.01
CHAMPION = 0.70


def decide(q_cbr):
    improved = q_cbr > CHAMPION + AB_DELTA_EPS
    if not improved:
        return "hold", False
    if q_cbr - CHAMPION < PRACTICAL_MARGIN:
        return "promote, but below the practical margin", True
    return "promote", True


for label, q_cbr in (
    ("tie", 0.70),
    ("+0.0000005", 0.7000005),
    ("+0.0005", 0.7005),
    ("+0.08", 0.78),
    ("-0.09", 0.61),
):
    verdict, promoted = decide(q_cbr)
    print(f"  q_cbr={q_cbr:<10} {label:<11} -> {verdict}")

print()
print(f"final champion quality: {0.70 + 0.08:.2f}")

Mechanism and limitations

Documented: the UCT rule, the fixture table and the champion promotion rule are in the chapter’s own text, and the code above is that text’s code. The production forms of both rules live in the reference implementation: the UCT value in stephanie/agents/dspy/mcts_reasoning.py and the promotion rule in stephanie/agents/dspy/memento.py.

Observed: the run above. The under-explored child scored 1.6038 and the leader 1.4141; the selection order was unvisited, under-explored, leader. The five promotion cases gave hold, hold, promote-below-the-margin, promote, hold, and the final champion quality was 0.78.

Not shown. The chapter’s own fixture directory, experiments/dspy-from-first-principles/ch21_beyond_book/, is not in the reference implementation’s history, so the page’s fixtures are the ones the chapter’s prose describes, not the chapter’s committed artefacts. The promotion rule here is written from the chapter’s description of it; the production method also performs the memory write, which is not exercised. The practical margin of 0.01 is a declared policy, not a measured one, and the chapter’s own point is that the epsilon alone is a floor and not a margin. Nothing here calls a model, so it says nothing about model quality or about whether a search with these settings finds anything useful.

Read the chapter .