Reasoning Models and Inference-Time Compute
Reasoning tokens, budget controls, observable results.
Chapter 32 examined API-level optimization. This chapter examines reasoning models that spend additional inference-time compute.
Reasoning tokens
Some models generate intermediate reasoning tokens before final answers. These tokens consume compute and cost but are not always visible.
Budget controls
Budgets limit reasoning length. Higher budgets can improve accuracy but increase latency and cost. The trade-off is task-dependent. More reasoning tokens do not always improve accuracy.
Research context
Inference-time compute includes serial reasoning, parallel samples, verifier search, and adaptive allocation. The cited sources study optimal test-time compute and DeepSeek-R1’s reasoning training. Claude’s extended thinking controls are model-specific and versioned; fixed budget_tokens is not universally supported.
The design rule is to measure accuracy, latency, and cost per solved task, and to reject the assumption that more reasoning always helps.
Observable results
We can measure accuracy, latency, and cost per solved task. We cannot claim access to hidden reasoning. The repository example uses a synthetic benchmark.
Experiment
The simulator ran 9 task/budget combinations with accuracy 0.556 and total cost 0.051. Tests verify solving, demo, and serialisation.
Limitations
- Synthetic tasks.
- No real model.
- No hidden reasoning access.
- No provider behavior.
- Controls are version-specific.
Key terms
- Reasoning tokens: intermediate computation.
- Budget: token limit.
- Inference-time compute: extra computation at inference.
- Accuracy: solved tasks.
- Cost per task: resource per solution.
Reader exercises
- Vary budget and plot accuracy.
- Add cost constraints.
- Explain why reasoning can increase latency.
- Describe how to evaluate hidden reasoning safely.
References
- Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters - research paper.
- DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning - technical report.
- Claude extended thinking - official provider documentation.
Implementation status
- Manuscript:
content/books/inference/33-chapter.md - Runnable example:
examples/ch33/ - Tests:
tests/test_ch33_reasoning.py - Evidence:
research/ch33/sources.mdandresearch/ch33/results.json - Verification command:
python -m unittest discover -s tests -v
The chapter is a draft. It does not claim real reasoning performance. The next chapter will examine tool calling.