Programmer.ieTechnical library for the AI era
  • Home
  • Books
  • Examples
  • Solutions
  • Learn
  • Prompts
  • Articles
  • Search
  • About
English简体中文

Evaluation

  • What Fine-Tuning Actually Changes
  • What Is Knowledge Distillation?
  • When Fine-Tuning Beats Prompting, RAG, Tools or Distillation
  • Distillation Versus Other AI Improvements
  • The Measurement Instrument
  • Write the Task Contract Before Training
  • Deriving the Distillation Objective
  • The First Search Baseline
  • Meat Proxy
  • A Baseline Can Defeat the Fine-Tune
  • Response-Based and Logit Distillation
  • The Memory Nexus
  • Dense Retrieval without Re-teaching Embeddings
  • Intelligence in the Wrong Direction
  • Examples Are Experimental Data
  • Feature-Based Distillation
  • You Cannot Optimize What You Cannot Measure
  • Attention and Relational Distillation
  • When the Metric Becomes the Target
  • Sequence-Level Distillation
  • Distillation From First Principles
  • Fine Tuning From First Principles
  • Semantic Failure Analysis and Hard Negatives
  • A Revolver, Not a Foundation
  • The Nearest Neighbor Can Be Wrong
  • Ensembles and Multiple Teachers
  • What Survived the Transformation?
  • The Data Behind the Knowledge
  • Retrieval From First Principles
  • Optimize the Instructions
  • From Agent Architecture to Agent Engineering
  • Constructing Teaching Tasks
  • A Resolved Promise Is Not a Correct Answer
  • Synthetic Data Generation
  • Evaluate the Feature, Not the Demo
  • Data Filtering and Curriculum Learning
  • Read the Product, Not the Code
  • Dataset Distillation
  • Appendix: The Evidence Ledger
  • Distilling Transformers and Instruction Models
  • Distilling Reasoning
  • Don't Let the Optimizer Cheat
  • Tool Choice Is a Behavioral Problem
  • Choose the Small Model and Measure Baselines
  • Bounded Adaptive Retrieval: When to Stop
  • Did the Bridge Preserve the Space?
  • Relevance Judgments and Test Collections
  • Metrics that Answer Different Questions
  • Controlled Experiments and Regression Gates
  • Can a Smaller Representation Preserve a Larger One?
  • Coding and Structured Generation
  • Machine Coding
  • Latency, Cost, and Operational Reliability
  • Does Better Retrieval Improve the Task?
  • Did the Context Help?
  • When the Process Learns
  • Evaluating Knowledge Transfer
  • Safety, Refusals and Reward Hacking
  • Evaluation That Measures the Task
  • Capstone II: The Language That Edited Its Own Book
  • Testing Stochastic Software
  • Measure agent reliability with pass@1 and pass^k
  • FineTuneLab: Run and Compare the Training Pipeline
  • Appendix B — Evidence and Claim Language
  • Appendix C — Experiments and Reproducibility
  • I Don't Write Code Anymore
  • Advanced Agents From First Principles 23: Your Infrastructure Is Healthy. Why Is the Agent Getting Worse? Detect Behavioral Drift and Roll Back Safely
  • Advanced Agents From First Principles 17: What Is Your Agent Actually Uncertain About?
  • Advanced Agents From First Principles 14: Can Your Agent Learn From Its Own Trajectories Without Learning the Wrong Lessons?
Programmer.ie

Technical library for the AI era

Understand what is changing. Learn how it works. Use it.

© 2026 Ernan Hughes
  • Home
  • Books
  • Examples
  • Solutions
  • Learn
  • Prompts
  • Articles
  • Search
  • About