The second edition of The ChatML (Chat Markup Language) Handbook is out, dated June 2026. It answers a question the first edition did not ask.
A fair reader in 2026 opens a book about ChatML and asks whether they are learning a fossil. ChatML was an OpenAI markup from 2023, and OpenAI's public APIs barely mention it now. The honest answer splits in two, and the second edition puts it in the preface rather than burying it.
The category won. The single standard never arrived. Structured, role-tagged turns
with explicit boundaries are now the interchange format of the whole open-model ecosystem.
vLLM, Text Generation Inference and Hugging Face chat templates all speak an ordered list
of role-tagged messages. Qwen2.5 and SmolLM2 ship the literal <|im_start|> and
<|im_end|> tags. What did not happen is one universal markup. Claude uses an XML-style
dialect of its own, and Hugging Face hosts a hundred-plus distinct chat_template.jinja
families rather than one canonical format.
So the book teaches the durable part: the grammar of roles, boundaries and ordering, rather than any one vendor's tag set. Learn it once and Qwen, Claude and your serving stack all read as variations on the same idea. Chapter 1, section 1.11 lays out what won and what fragmented, with the evidence. If you want the same argument as a standalone piece with primary sources, I wrote ChatML: What It Is, Why OpenAI Removed It, What Replaced It.
Buy on Amazon: United States | India
What Changed in the Second Edition
An honest relevance check. The preface and Chapter 1 section 1.11 address head-on what won and what fragmented in ChatML's adoption. This replaced a first-edition section that overclaimed.
A support bot that actually runs. Part III is now a dedicated project: Support Bot v3.4, built on ChatML with FastAPI, Ollama, Qdrant and Jinja2. It is a full implementation rather than a walkthrough of fragments.
New chapters on tool execution and memory persistence. Chapter 8 covers the tool-execution layer and Chapter 9 covers long-term conversational memory with vector storage.
A new appendix on today's provider APIs. Appendix E covers streaming, multimodal messages, structured outputs, prompt caching and reasoning models. Its thesis is the same as section 1.11: these extend the message list rather than replacing it.
Chat templates are foregrounded. Every model on the Hugging Face Hub ships a Jinja
template in its tokenizer_config.json, and tokenizer.apply_chat_template(messages)
turns your message list into the exact token sequence that model was trained on. ChatML is
the mental model. The chat template is the machinery that runs it on real weights.
Standard front and back matter. Conventions, a guide to using the code examples, further reading with real links, an epilogue and an errata page.
Why I Wrote This Book
Over the last few years, Large Language Models (LLMs) have moved from research systems to the platforms behind customer support, automation, copilots and agent ecosystems.
One reality has stayed constant:
Most developers know how to use LLMs, but very few know how to control them.
Every AI engineer I meet struggles with inconsistent model behavior, fragile prompts, unexpected reasoning, and tools that “sometimes work.” The missing piece is structure.
Unstructured prompts make the machine guess where the boundaries are. A guess is something an attacker controls and a test cannot pin down. That is the one-sentence case for structured prompting, and it is the sentence the book hands you for a design review.
What the Book Covers
Eleven chapters, five appendices, and a working project. Inside, you’ll find:
Part I - Foundations of ChatML
Chapter 1: The Evolution of Structured Prompting - From Early Chatbots to the Architecture of ChatML
Chapter 2: Anatomy of a ChatML Message - Understanding <|im_start|> and <|im_end|> Boundaries, Role Tags, and Content Flow
Chapter 3: Roles and Responsibilities - System, User, Assistant, and Tool Roles - Maintaining Conversational Integrity
Chapter 4: Context and Continuity - How Memory and Context Persistence Enable Multi-Turn Dialogue
Chapter 5: Design Principles of ChatML - The Philosophy Behind Structure, Hierarchy, and Reproducibility in Communication
Part II - Engineering with ChatML
Chapter 6: Building a ChatML Pipeline - Structuring Inputs, Outputs, and Role Logic in Code
Chapter 7: Rendering with Templates - Using Jinja2 for Modular and Dynamic ChatML Message Generation
Chapter 8: Tool Invocation and Function Binding - Designing a Tool-Execution Layer for Reasoning and Automation
Chapter 9: Memory Persistence Layer - Building Long-Term Conversational Memory with Vector Storage and Context Replay
Chapter 10: Testing and Observability - Techniques for Evaluating Structured Prompts, Logging, and Reproducibility
Part III - The Support Bot Project
Chapter 11: Building a Support Bot Using ChatML - From Structured Prompts to Full AI Workflows
Part IV - Appendices (Ecosystem & Reference)
Appendix A: ChatML Syntax Reference - Complete Markup Specification and Role Semantics
Appendix B: Integration Ecosystem - How ChatML Interacts with LangChain, LlamaIndex, and Other Frameworks
Appendix C: Template and Snippet Library - Ready-to-use ChatML Patterns for Various Conversational Tasks
Appendix D: Glossary and Design Checklist - Key Terminology, Conventions, and Best Practices
Appendix E: ChatML and Today’s Provider APIs - Streaming, Multimodal, Structured Outputs, Caching, and Reasoning Models
What Makes This Book Unique?
There are many books on prompt engineering, but this one is different.
- Developer-Centric: Written for engineers, architects, and builders, not casual prompt users.
- Structured Prompting Over Guesswork: Moves away from “try this magic prompt” toward repeatable engineering patterns.
- 100% Practical: Diagrams, schemas, real tool-call examples, and ChatML templates you can paste directly into your code.
- Code That Compounds: The book builds on a typed
Messageand a swappablerender_chatml, not on hard-coded<|im_start|>literals scattered through your code. A function introduced in Chapter 1 is reused by name in Chapter 9, never re-implemented. - Vocabulary You Can Cite: Twelve named concepts are coined and collected in Appendix D, among them the replay tax, the constitutional layer, the ChatML thin waist, the side-effect boundary and the assertion line.
Who Should Read This Book?
This book is ideal for:
-
AI engineers & developers
-
Startup founders building with LLMs
-
Product teams adopting conversational UX
-
Researchers designing agent systems
-
Anyone serious about mastering structured prompting
If your job involves LLMs, you will benefit.
Why ChatML Matters Today
As LLMs become more capable, the bottleneck is no longer the model. It is how we talk to the model.
The role set grew instead of breaking. The original system / user / assistant triad
absorbed tool calls and tool responses for agentic work, then media content for images and
audio, without changing its shape. That elasticity is why the category survived while the
tags did not.
What the shape gives you:
-
Defines roles
-
Clarifies intent
-
Preserves context
-
Enables tool-use
-
Makes prompts deterministic
-
Extends to multimodal content without changing shape
Every serving stack assumes a role-based message list. The list of role and content objects is the contract. The tokens each model emits underneath it are an implementation detail. Learn the contract and the dialects stop mattering.
Get the Book
Second Edition, Kindle, June 2026.
United States: https://www.amazon.com/dp/B0G2GM44FD
India: https://www.amazon.in/dp/B0G2GM44FD
If you find value in the book, an Amazon review helps it reach more AI builders.
Related Articles
Natural Language Processing Nlp
- How Google's SynthID Actually Works: A Visual Breakdown
- On Emergent Abilities of Large Language Models
- Provenance in AI: Tracking AI Lineage with Signed Provenance Logs in Python - Part 2
Follow for more technical deep dives on AI/ML systems, production engineering, and building real-world applications:
