Claude, Caching and API-Level Optimization

Public API behavior versus unknown internals.

Inference From First Principles Chapter 32 of 45

01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45

Chapter 31 examined protocols. This chapter examines API-level optimization, using public documentation only.

Public API versus internals

Provider documentation describes request/response behavior, not serving internals. We must not infer proprietary implementations from API behavior.

Prompt caching

Prompt caching allows reuse of previously processed prefixes. Eligibility depends on documented fields such as minimum token counts and cache control markers. Savings are documented discounts, not measured system behavior.

Research context

Anthropic’s documentation defines prompt caching as an observable contract: cache breakpoints, TTLs, ordering, and usage accounting are visible. The Messages API remains stateless: the client still resends context, while provider-side caching may reduce cost. Pricing is published but volatile and contract-sensitive.

The boundary is clear: API-visible cache fields are not access to internal KV layout. Model cache economics using dated prices and real reuse distributions.

Cost

API cost is determined by published pricing. Optimization should be measured at the application level, not assumed from provider internals.

Experiment

The repository mock evaluates cache eligibility based on token count. A 1500-token message was eligible. Tests verify eligibility, ineligibility, and serialisation.

Limitations

Mock only.
No real API calls.
No cost measurement.
Internals unknown.
Pricing may change.

Key terms

Prompt caching: reuse of processed prefix.
Cache control: documented marker.
Eligibility: whether caching applies.
API-level optimization: application-side improvements.
Unknown internals: not inferable.

Reader exercises

Add cost calculation from published pricing.
Test minimum token threshold.
Explain why API behavior is not implementation proof.
Design an experiment that respects provider terms.

References

Claude prompt caching - official provider documentation.
Anthropic Messages API - official provider documentation.
Anthropic API pricing - official provider documentation.

Implementation status

Manuscript: content/books/inference/32-chapter.md
Runnable example: examples/ch32/
Tests: tests/test_ch32_cache.py
Evidence: research/ch32/sources.md and research/ch32/results.json
Verification command: python -m unittest discover -s tests -v

The chapter is a draft. It does not claim provider internals. The next chapter will examine reasoning models.