http://programmer.ie/tags/lm-judges/ Recent content in LM Judges on Programmer.ie Hugo en-us Fri, 28 Aug 2026 10:40:00 +0100 http://programmer.ie/books/dspy-from-first-principles/09-chapter/ Fri, 28 Aug 2026 10:40:00 +0100 http://programmer.ie/books/dspy-from-first-principles/09-chapter/ <h1 id="the-metric-is-part-of-the-program">The Metric Is Part of the Program</h1> <p>Chapter 8 gave us a baseline harness. It deliberately used a weak smoke-test metric because the first job was to make evaluation run.</p> <p>Now the dangerous part begins.</p> <p>During optimization, DSPy searches for program states that score well under the metric we provide. The optimizer does not know whether the score is a faithful proxy for the behavior we actually want.</p> <hr> <h2 id="1-the-metric-ladder">1. The metric ladder</h2> <p>Different tasks tolerate different metrics.</p>