MangaGo mistook a preorder for an already-released volume

“I built it with Claude Code” — some people hear that and picture something like microwaving a meal, or pressing a button on a vending machine, assuming the thing they want pops right out. The world isn’t that easy. Hand over a spec, and something that runs does come out, sure. But getting it to the point where you can actually use it every day still takes real effort.

That’s just what making things is.

The app that went looking for Volume 17

MangaGo, which I wrote about in “I built an app just for myself that tells me when manga comes out,” was no exception.

When I registered If Only We Could Throw Away Such a Family, it came back with “Next: Vol. 17 · TBA.”

Volume 16 ships on October 14, and bookstores are already taking preorders. The next volume should be Volume 16 — Volume 17 hasn’t even been announced yet. Yet the app skipped right past Volume 16 and dutifully went looking for the release date of a Volume 17 that doesn’t exist, found nothing of course, and dutifully reported back “TBA.” It even tacked on an “Unverified” label for good measure.

Why was it off by one volume?

When you register a title, this app has Claude (Haiku 4.5) run a web search to count “how many volumes have come out so far,” then goes looking for the release date of the next one. The fact that it was searching for Volume 17 means it had already miscounted by one at the counting stage.

The first theory, offered by the very Claude I’d enlisted to help build this, was that “the print volumes probably only go up to Volume 12, and 16 is likely the count of individually-released digital chapters.” It sounded plausible, but it was a conclusion offered without verification. When I went back and checked the print volume listing on Rakuten Books, print Volume 15 had come out on May 14, and Volume 16 was slated for October 14 and open for preorder. So much for the print-versus-digital miscount theory.

The model had been counting Volume 16, which was still on preorder, as “already released.”

So I added a line to the prompt used when registering a title: “number of print volumes (excluding digital and single-chapter releases).” I tried it three times: 15, 16, 16. Didn’t fix it. The model had never actually been confusing print and digital in the first place, so hammering on that point having no effect was, in hindsight, about what you’d expect.

Next, I passed today’s date into the prompt and instructed it to count “only what has been released as of today (excluding preorders).” The results were 15, 15, 15.

The model doesn’t know what today’s date is.

Thinking about it, the word “already released” only means anything relative to some notion of “today.” A bookstore page turned up by a web search lists Volume 16 right there as a product, with a preorder button attached. Without knowing what today’s date is, there’s no way to tell whether that’s a released book or a preorder. A human would unconsciously check it against a calendar; the model simply had nothing to check it against.

Alongside this, I added an “Edit” feature to the app so the volume count could be corrected by hand. After fixing it to 15 and rechecking, it correctly pulled “Vol. 16 · Out Wed, Oct 14.”

Title detail view. Added the ability to correct the current volume count

To be honest, similar things kept happening before I got to this point too.

The cheaper Haiku actually performed better

Before I started building, I measured whether Haiku 4.5 would be enough, or whether I’d need Sonnet 5.5. I had Opus draft the ground-truth data, attached two sources — the publisher and a bookstore — to each entry, and had a human check only the dubious ones. If whoever builds the ground-truth data fabricates dates, the whole measurement falls apart, so I was careful here.

The result: the cheaper Haiku did better. Accuracy on release dates was 71% for Haiku versus 48% for Sonnet; the rate of catching an author’s new releases was 47% versus 7%; cost per run was about $0.06 for Haiku against about $0.14 for Sonnet; and time was 6.7 seconds versus 21.8 seconds. In both cases, every wrong answer came back as an honest “date unknown” — not a single one fabricated a date that didn’t exist. That much, at least, I’m grateful for.

The reason Sonnet’s author-check rate fell to 7% became clear from the logs. Sonnet was using the newer web search tool, which bundles five search terms into a single code execution. Since I’d capped the number of searches at four per run to control costs, Sonnet burned through its entire budget on the very first move and returned an empty array without finding anything. It was a mismatch between how Sonnet searches and the cap I’d set — loosening the cap might well change the results.

Adding an instruction made it report a completed manga as a new release

Haiku’s author-check was missing things like the first volume of a new series, or books where the author was credited as the original creator, so I added explicit instructions to look for those, and raised the search cap from 4 to 5. The catch rate stayed flat at 47%, while the cost went up by 30%. On top of that, it reported N to S, a completed series (8 volumes total), as a new release all three times. The volume number was different each time, and the date belonged to some other title entirely. In the app, that would have meant a book that doesn’t exist showing up on the calendar, complete with a notification.

That one wasn’t funny, so I reverted it immediately.

Author registration still isn’t fully worked out

Volume 14 of Masakazu Ishiguro’s Dai Dark [Note: the registered author entry], which I’d registered under the author, failed to turn up in all three checks. Haiku ran a search with what looked like a reasonable query, then returned nothing. So I’m not placing too much trust in author registration — for any title I actually want to track reliably, I register it as a title in its own right too.

What didn’t come out of the microwave

Within two days of handing over the spec, the app was genuinely working. In microwave terms, it’s fair to say this much did come out with a quick zap.

But nobody noticed that it was counting a not-yet-released Volume 16 as already out, until I registered the title myself and felt something was off about the “Volume 17” on screen. The first plausible-sounding theory for the cause turned out to be wrong once I checked it. That an instruction I’d added with good intentions would make it report a completed manga as a new release — I didn’t know that either, until I measured it three times. Whether cheap Haiku would be enough, or whether I’d need Sonnet — I didn’t know that until I measured it.

Claude Code did almost all of the hands-on fixing. Even so, noticing what feels off once you actually use the thing, not taking an explanation at face value and checking it instead, deciding to revert after looking at what you measured — these three things don’t come out just from pressing a vending machine button.

This, after all, is what making things is.

This piece was planned and directed by Yuichiro Kuzuryu and drafted by AI.

Originally published in Japanese at https://clazytech.com/2026/10/1871/. Translated with LLM assistance and reviewed before publication.