<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom">
	<channel>
		<title>models-from-first-principles examples on Programmer.ie: Modern AI programming</title>
		<link>http://programmer.ie/examples/models-from-first-principles/</link>
		<description>Recent content in models-from-first-principles examples on Programmer.ie: Modern AI programming</description>
		<generator>Hugo</generator>
		<language>en-US</language>
		
		
		
		
			<atom:link href="http://programmer.ie/examples/models-from-first-principles/index.xml" rel="self" type="application/rss+xml" />
			<item>
				<title>One model, four sub-models, and one shared tensor</title>
				<link>http://programmer.ie/examples/models-from-first-principles/01-chapter/</link>
				<pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate>
				<guid>http://programmer.ie/examples/models-from-first-principles/01-chapter/</guid>
				<description>&lt;p&gt;The word model suggests one monolithic object, and the code usually tells a&#xA;different story: a complicated model is a collection of smaller models, each a&#xA;collection of layers. Opening an unfamiliar model means descending that&#xA;hierarchy until the tensors are visible.&lt;/p&gt;&#xA;&lt;p&gt;This program builds the chapter&amp;rsquo;s multi-head evaluator and the recurrent&#xA;block from it: the shape at every stage of the diagram, the parameter count&#xA;of every part, and what six applications of a block cost.&lt;/p&gt;</description>
			</item>
			<item>
				<title>Two embeddings in, one number out, and what gives the number meaning</title>
				<link>http://programmer.ie/examples/models-from-first-principles/02-chapter/</link>
				<pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate>
				<guid>http://programmer.ie/examples/models-from-first-principles/02-chapter/</guid>
				<description>&lt;p&gt;An architecture that maps a pair of embeddings to a scalar says nothing about&#xA;what that scalar means. The meaning comes from the objective, the data and&#xA;the evaluation, not from the name of the output.&lt;/p&gt;&#xA;&lt;p&gt;This program trains the chapter&amp;rsquo;s pairwise model on a synthetic task, then&#xA;asks the chapter&amp;rsquo;s question: is the network better than the cosine similarity&#xA;that needs no training?&lt;/p&gt;&#xA;&lt;h2 id=&#34;run-it&#34;&gt;Run it&lt;/h2&gt;&#xA;&lt;p&gt;Save this as &lt;code&gt;mrq_quality_model.py&lt;/code&gt; and run it with Python 3.11 or later. It&#xA;needs PyTorch: &lt;code&gt;pip install torch&lt;/code&gt;, then &lt;code&gt;python mrq_quality_model.py&lt;/code&gt;.&lt;/p&gt;</description>
			</item>
			<item>
				<title>The optimizer is a model too: two averages and a preconditioner</title>
				<link>http://programmer.ie/examples/models-from-first-principles/08-chapter/</link>
				<pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate>
				<guid>http://programmer.ie/examples/models-from-first-principles/08-chapter/</guid>
				<description>&lt;p&gt;An optimizer is a model of sorts: it keeps state, observes gradients,&#xA;transforms them and decides how far each parameter moves. PACS keeps two&#xA;statistics per parameter, a mean gradient and a mean squared gradient.&lt;/p&gt;&#xA;&lt;p&gt;This program builds PACS from the chapter&amp;rsquo;s update equations, solves a scalar&#xA;quadratic with it, then hands it two coordinates whose gradients differ by a&#xA;thousandfold.&lt;/p&gt;&#xA;&lt;h2 id=&#34;run-it&#34;&gt;Run it&lt;/h2&gt;&#xA;&lt;p&gt;Save this as &lt;code&gt;pacs_optimizer.py&lt;/code&gt; and run it with Python 3.11 or later. It&#xA;needs PyTorch: &lt;code&gt;pip install torch&lt;/code&gt;, then &lt;code&gt;python pacs_optimizer.py&lt;/code&gt;.&lt;/p&gt;</description>
			</item>
	</channel>
</rss>
