Inference Protocols and Harmony

Tokenized message envelopes, roles, channels, tool handoffs.

Inference From First Principles Chapter 31 of 45

01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45

Chapter 30 handled reliability. Part 6 now examines protocols and application-level optimization, starting with message envelopes.

Message envelopes

Inference protocols wrap raw text in structured messages with roles and channels. This allows system, user, assistant, and tool messages to be distinguished.

Roles and channels

Roles identify who produced a message. Channels distinguish analysis from final output. This separation supports reasoning and safety.

Tool handoffs

Tool calls require structured requests and responses. The protocol must preserve boundaries so untrusted content is not interpreted as instructions.

Research context

Harmony is a model-facing protocol contract, not a general transport protocol. The OpenAI Harmony sources define roles, channels, tools, and token formatting for compatible models. Conformance checks verify that a backend preserves the protocol rather than merely returning text.

The key boundary is between protocol formatting, HTTP/SSE transport, and application tool execution. Keeping these separate prevents untrusted content from becoming instructions.

Experiment

The repository example parses a small Harmony-like envelope into 4 messages with no errors. Tests verify parsing, unparsed lines, demo, and serialisation.

Limitations

Toy parser.
No real Harmony runtime.
No tool execution.
No security audit.
Harmony is model-specific and must be pinned to a model revision.

Key terms

Envelope: structured message wrapper.
Role: message producer.
Channel: content category.
Tool handoff: structured tool interaction.
Boundary: separation of trusted/untrusted content.

Reader exercises

Add tool-call messages.
Test malformed envelopes.
Explain why channels matter for safety.
Compare Harmony to chat completions.

References

openai/harmony - first-party protocol repository.
OpenAI Harmony response format - official provider guide.
Verifying gpt-oss implementations - official provider guide.

Implementation status

Manuscript: content/books/inference/31-chapter.md
Runnable example: examples/ch31/
Tests: tests/test_ch31_harmony.py
Evidence: research/ch31/sources.md and research/ch31/results.json
Verification command: python -m unittest discover -s tests -v

The chapter is a draft. It does not claim protocol compliance. The next chapter will examine Claude caching.