http://programmer.ie/tags/interpretability/ Recent content in Interpretability on Programmer.ie: Modern AI programming Hugo en-us Mon, 10 Aug 2026 20:41:00 +0100 http://programmer.ie/post/cellular-automata-from-first-principles-48/ Mon, 10 Aug 2026 20:41:00 +0100 http://programmer.ie/post/cellular-automata-from-first-principles-48/ <h1 id="cellular-automata-from-first-principles-48-what-did-the-neural-ca-actually-learn">Cellular Automata From First Principles 48: What Did the Neural CA Actually Learn?</h1> <p>Visualization gives us hypotheses.</p> <p>Intervention gives us evidence.</p> <p>If we suspect a hidden channel carries useful information, the next question is simple:</p> <div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-text" data-lang="text"><span style="display:flex;"><span>what happens if we remove it? </span></span></code></pre></div><p>This chapter turns NCA interpretability into controlled experimentation.</p> <hr> <h2 id="establish-the-behavioral-baseline">Establish the behavioral baseline</h2> <p>Before touching the model, record its normal performance.</p> <div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-python" data-lang="python"><span style="display:flex;"><span>baseline <span style="color:#f92672">=</span> evaluate_suite(model, test_mazes) </span></span></code></pre></div><p>Keep metrics such as:</p> <div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-text" data-lang="text"><span style="display:flex;"><span>valid-path rate </span></span><span style="display:flex;"><span>optimal-path rate </span></span><span style="display:flex;"><span>mean excess path length </span></span><span style="display:flex;"><span>unreachable-maze accuracy </span></span><span style="display:flex;"><span>stabilization time </span></span></code></pre></div><p>Every intervention will be compared with this baseline.</p> http://programmer.ie/post/cellular-automata-from-first-principles-47/ Mon, 10 Aug 2026 20:40:00 +0100 http://programmer.ie/post/cellular-automata-from-first-principles-47/ <h1 id="cellular-automata-from-first-principles-47-inspect-hidden-state-propagation">Cellular Automata From First Principles 47: Inspect Hidden-State Propagation</h1> <p>A neural cellular automaton can solve a maze while most of its computation remains invisible.</p> <p>The visible output might be one distance channel.</p> <p>The internal state may contain twelve more channels evolving underneath it.</p> <p>This chapter asks:</p> <blockquote> <p>what is moving through those hidden channels while the answer is being computed?</p></blockquote> <hr> <h2 id="capture-the-entire-state-trajectory">Capture the entire state trajectory</h2> <p>Instead of saving only the final output:</p>