The Universe of Discourse

Documentation is a message in a bottle

Something interesting happened at work this week. Our company is going to a convention later this month, and they will have a booth with big TV screens showing statistics that update in real time. My job is to write the backend server that delivers the statistics.

I read over the documents that the product people had written up about what was wanted, asked questions, got answers, and then turned the original two-line ticket into a three-page ticket that said what should be done and how. I intended to do the ticket myself, but it's good practice to write all this stuff down, for many reasons:

• Writing things down forces me to think them through carefully and realize what doesn't make sense or what I still don't understand.

• I forget things easily and this will keep the plan where I can find it.

• I might get sick, and if someone else has to pick up the project this might help them understand what I was doing.

• If my boss gets worried that all I do is post on 4chan all day, this is tangible work product that proves I did something else that might have enhanced shareholder value.

• If I'm tempted to spend the day posting on 4chan, and then to later claim I spent the time planning the project, I might fool my boss. But without that tangible work product, I won't be able to fool myself, and that's more important.

• Conversely if I later think back and ask “What was I doing the week of March 2?” I might be tempted to imagine that all I did was post on 4chan. But the three pages of ticket description will prove to me that I am not just a lazy slacker. This is a real problem for me.

• In principle, a future person going back to extend the work might find this helpful documentation of what was done and why. Does this ever really happen? I don't know, but it might.

• I like writing because writing is fun.

A few days after I wrote the ticket, something unexpected happened. It transpired that person who was to build the front-end consumer of my statistics would not be a professional programmer. It would be the company's Head of Product, a very smart woman named Amanda. The actual code would be written by Claude, under her supervision.

I have never done anything like this before, and I would not have wanted to try it on a short deadline, but there is some slack in the schedule and it seemed a worthwhile and exciting experiment.

Amanda shared some screencaps of her chats with Claude about the project, and I suggested:

When you get a chance, please ask Claude to write out a Markdown file memorializing all this.  Tell it that you're going to give it to the backend programmer for discussion, so more detail is better. When it's ready, send it over.

Claude immediately produced a nine-page, 14-part memo and a half-page overview. I spent a couple of hours reviewing it and marking it up.

It became immediately clear that Claude and I had very similar ideas about how the project should go and how the front and back ends would hook up. So similar that I asked Angela:

It looks like maybe you started it off by feeding it my ticket description.  Is that right?

She said yes, she had. She had also fed it the original product documents I had read.

I was delighted. I had had many reasons for writing detailed ticket descriptions before, but the most plausible ones were aimed back at myself.

The external consumers of the documentation all seemed somewhat unlikely. The person who would extend the project in the future probably didn't exist, and if they did they probably wouldn't have thought to look at my notes. Same for the hypothetical person who would take over when I got sick. My boss probably isn't checking up on me by looking at my ticketing history. Still, I like to document these things for my own benefit, and also just in case.

But now, because I had written the project plan, it was available for consumption when an unexpected consumer turned up! Claude and I were able to rapidly converge on the design of the system, because Amanda had found my notes and cleverly handed them to Claude. Suddenly one of those unlikely-seeming external reasons materialized!

On Mastodon I keep seeing programmers say how angry it makes them that people are willing to write detailed CLAUDE.md and PROJECT.md files for Claude to use, but they weren't willing to write them for their coworkers. (They complain about this as if this is somehow the fault of the AI, rather than of the people who failed in the past to write documentation for their coworkers.)

The obvious answer to the question of why people are willing to write documentation for Claude but not for their coworkers is that the author can count on Claude to read the documentation, whereas it's a rare coworker who will look at it attentively.

Rik Signes points out there's a less obvious but more likely answer: your coworkers will remember things if you just tell them, but Claude forgets everything every time. If you want Claude to remember something, you have to write it down. So people using Claude do write things down, because otherwise they have to say them over and over.

And there's a happy converse to the complaint that most programmers don't bother to write documentation. It means that people like me, professionals who have always written meticulous documentation, are now reaping new benefits from that always valuable practice.

Not everything is going to get worse. Some things will get better.

[Other articles in category /tech/gpt] permanent link