Skip to content

Conversation Is the Control Layer

A practitioner's field guide to designing AI agents that listen, wait and act.

Every claim is traceable. Every number has a source. Every example says whether it really happened.

The design object has moved. For most of the history of conversational products we designed utterances, and we were right to, for as long as the system did one thing at a time in response to one input. That era is over.

The systems we build now listen continuously, choose tools, move between voice and screen, and act on our behalf with other people. The behaviour of the system is mostly not in the response — it is in the timing, the restraint, the permission, the modality choice, the silence, and the moment of disclosure.

So we have to specify that behaviour. Which means writing down things that used to be left as vibes.

This book is about how. It is written for anyone who has to specify agent behaviour: conversation designers, product managers, agent designers, and the full-stack builder who is all three at once.

Free, open, and updated monthly. ~70,000 words. No sign-up, no email required to read.

Start here

The book is reference material with an argument in it, and it is not meant to be read front to back. Pick your route by the question you are actually answering.

If your question is… Read Why
The person it happens to Ch 3 · Ch 4 · Ch 10 What it feels like to talk to your agent, and the metric that catches your failures
The promise it carries Ch 6 · Ch 12 · Ch 14 Authority, permission, the anti-patterns that get institutionalised in backlogs, and the launch threshold
The mechanism that acts Ch 2 · Ch 9 · Ch 17 The six-layer model, the coordination table, and the most directly implementable artefact in the book

Three questions, one decision. Every behaviour decision has three objects of responsibility — the person on the other end of it, the promise the product has made, and the mechanism that will act on it. They are three different questions about the same decision, and a decision is incomplete until all three have been answered.

If you only read one chapter, read Chapter 4. If you only work one thing, work a drill.

A diagnostic. Each policy chapter ends with the same three questions answered. Read one, and notice which of the three answers you want to skip. A team will usually have one of the three genuinely owned, one partially owned, and one owned by nobody. The one nobody owns is where your next incident comes from.

What's in it

Part Chapters What it covers
I — The shift 1–2 Why the design object moved, and the six-layer model
II — The six policies 3–8 Language · Conversation · Modality · Agency · Attention · Representation
III — Coordination 9–10 The orchestration layer, coherence cost, the turn taxonomy
IV — The working library 11–12 Sixteen patterns, twelve anti-patterns
V — Practice 13 Twenty-two drills, all answered
VI — Evaluation and failure 14–15 Five dimensions and an error taxonomy; the failures your dashboard cannot show you
VII — With a team 17 Four artefacts to fill in, a maturity model, a critique method
Front matter — Preface, how to use this book, which question are you answering
Back matter — Reading ledger, drill index, bibliography, index

The chapters

# Chapter Words
1 The design object moved 2,831
2 Conversation is a control layer, not the interface 2,514
3 Language policy: what should I say? 2,850
4 Conversation policy: when should I speak, listen, interrupt or wait? 3,558
5 Modality policy: should I say it, show it or act? 2,572
6 Agency policy: should I act or ask permission? 3,339
7 Attention policy: what should I surface while I'm working? 2,692
8 Representation policy: who am I when I interact on your behalf? 3,080
9 Coordination policy: the orchestration layer 2,731
10 Continuity, coherence cost, and meaning-then-behaviour 2,822
11 Sixteen patterns 4,837
12 Twelve anti-patterns 3,413
13 Twenty-two drills 14,453
14 Evaluating behaviour 3,725
15 The silent dropout problem 3,947
17 Contracts, checkpoints and maturity 5,465
Front matter 2,278 Preface · How to use · Which question?
Back matter 3,076 Reading ledger · Drill index · Bibliography · Index

Chapter 16, a set of real-project case studies, was cut. The reasoning is in the production plan in the repository — briefly: the material sat behind non-disclosure agreements, and the book's argument does not need anecdotes to hold.

What makes this different

Every research claim has a resolvable identifier. The reading ledger lists every external source the book leans on — arXiv IDs, DOIs, URLs — what it found, and what it was doing in the argument. All identifiers were checked when the edition was published. If you disagree with a number, this is where you start.

The frameworks are presented as hypotheses, not findings. The book makes ten claims it is only willing to claim, and they are listed as such in the back matter. A book that presented them as settled would be lying in a way that is easy to do and hard to detect.

The examples declare themselves. Some case studies come from published research, some from the drill exercises, and some are composites written to make a point. Every one is labelled. There are no unverifiable anecdotes, and there are no client stories that could not be published.

The failures are specific. In a study of 100,000 real conversations, 63% contained a failure and 79% of those failures were invisible — the user never reacted. Better models have changed the rate of failure and not the shape of it. Chapter 15 is about designing for the failures your dashboard is structurally unable to show you.

The five things you can actually take away

  • The drill index — twenty-two drills, indexed three ways: by cluster, by policy, and by what you are trying to do in the hour you have. If you have one hour, the answer is three specific drills.
  • The turn and state specification — an eight-column table that maps the states your system can be in and what the user can do from each one. Most teams that fill one in for the first time discover they did not agree about what "processing" means.
  • The interaction contract — seven sections, one page, and the unfilled fields are the finding: a blank "must never do" line tells you nobody has decided what the system must not do.
  • The autonomy decision card — seven fields and six levels, because "it had permission to do that" is not a defence.
  • The maturity model — four stages, with the diagnostic that tells you which one you are actually in.

How this stays true

This is a living book. It gets updated when the underlying claims change or when someone points at something wrong — not because it is the month to do a release.

That rule is the whole editorial policy, and it is deliberately stricter than a monthly cadence:

  • Chapters change when their claim changes. If a paper is revised, a benchmark is superseded, or an argument here turns out to be wrong, the chapter changes and the git history says why.
  • The spine is stable. The sixteen chapters and the three routes are fixed. This book does not grow a new chapter every quarter, because a book that keeps acquiring chapters stops being a book.
  • Sources are checked, not trusted. Every identifier in the reading ledger was resolved when the edition was published. Dead links get fixed or the citation gets cut.
  • The drill answers stay honest. Eleven of the twenty-two are worked answers — positions that survived an argument. The other eleven are strong answers, written from markers and openly open to criticism. The book says which is which.

The commit history is the changelog. It shows what was claimed, when, and what changed — including the parts that were wrong.

STATUS.md is the standing version record: what was last reviewed, what the monthly check actually does, and what deliberately does not change.

Read it

Everything is Markdown, in the repository, and the site is generated from it.
Front matter Read — preface, how to use this book, which question are you answering
Chapters 1–17 Start from the routing table above
Back matter Read — reading ledger, drill index, bibliography, index
Drills Chapter 13 — all twenty-two, with answers

Who this is for

If you are designing a voice agent, an AI assistant, or any system that acts on someone's behalf — this book is for you.

If you are here for a prompt guide, a model comparison, or a voice-acting manual, this is not your book. The book says so in the preface, and it means it. The frameworks are deliberately about behaviour rather than capability, because capability ages and behaviour does not — though the specific latency figures and benchmark numbers will date, and you should read them as evidence that was true when it was written.

If you are reviewing someone else's agent work, start with Chapter 17's critique method and the ten questions to ask before shipping.

Contributing

Corrections are welcome and treated as contributions, not nitpicks. If a number is wrong, a source is dead, or an argument does not hold, open an issue or send a pull request.

If you disagree with a strong drill answer — the eleven written from markers rather than argued — that is the most useful kind of contribution, and the reason they are labelled.

Working notes, editorial decisions and open questions about the book's own argument live in editorial/ in the repository and are not part of the manuscript.

The book checks itself

Three scripts run over the manuscript. They are the reason to trust the tables above.

python3 scripts/check-links.py     # every relative link resolves to a real file and anchor
python3 scripts/check-counts.py    # every chapter header states its measured word count
python3 scripts/check-drills.py    # every drill-index claim is backed by a real citation

The third one is the important one. The drill index sends readers to chapters that teach each drill — and a claim that is not backed by a real citation sends someone to a chapter that does not teach the thing they came for. Building the index found seven such overclaims; all are now fixed.

About the author

Ian Roberts — designer and builder working on agent behaviour, voice interfaces and evaluation. Previously across conversation design, product and engineering, which is why this book keeps insisting that the three questions belong to one person even when they belong to three.

Writing this book is also how the work gets better. The book is the thinking made public so it can be argued with.

Licence

Text and diagrams © Ian Roberts, licensed under CC BY 4.0. Quote it, translate it, teach from it, adapt it — with attribution and a link back.

Edition 0.1 · 14 November 2026 · 70,183 words · updated monthly