The problem
The client has a small content team and a long list of landing pages. A general writing tool produced a first draft in minutes and a statistic in the same breath. Some of those statistics were from older campaigns, some from a competitor's page, and some from nowhere the team could find. Legal review was becoming a hunt for the number, which erased the time the draft had saved.
The editor's rule was simple and had not been enforced in the tool: if we did not approve the source, the number does not go on the page. The project was to make a draft that could follow that rule without the editor pasting sources in afterward.
Why the source list is the brief
A style guide in the prompt does not stop a model from recalling a statistic it saw in training. The draft is built from a source packet the client maintains: approved studies, internal metrics with an owner, and claims legal has already accepted. If a sentence needs a number that is not in the packet, the sentence is omitted, not approximated.
Architecture
The packet is the only factual input
Each page request names the product, the audience, and the packet. The draft may use wording around those sources. It may not introduce a figure, a customer count, or a performance claim from outside them.
Every figure in the draft carries a source id
The structured output attaches the packet id to each number. A number with no id fails the check and the draft is regenerated once without that sentence. If it still fails, the draft goes to the editor with the sentence removed and a note that the packet had no source.
Unapproved claims are listed, not smuggled
The editor sees a short list of claims the brief seemed to want that the packet could not support. That list is how the team decides to commission a source, rather than discovering the gap on the live page.
Publish is still the editor
The draft lands in the CMS as a draft. The editor publishes. The check is on what reaches the editor, not on replacing the editor.
Challenges and tradeoffs
- Internal metrics had no owner in the packet. A number from a dashboard was pasted in without a date. We required an owner and an as-of date on every internal metric, and the draft prints the date next to the figure so a six-month-old count is visible.
- The model restated a sourced number more strongly than the source. A source said 'up to' and the draft dropped the hedge. The check now compares the figure and the qualifier, and a stronger claim fails the same way a missing source fails.
- Editors bypassed the tool for 'just a headline.' Headlines were where unsourced claims had been sneaking back in. The headline is in the same structured output and the same check. A headline with a number needs the same id.
Results
On the pilot pages, every draft that reached review had a source id on every figure, and reviewers found no statistic that was not in the packet. Claims the packet could not support were in the gap list, not in the copy. An editor still published each page.
Drafts got shorter where the packet was thin. The team treated that as a brief problem, which is what it was.
What we'd do differently
We would have put the qualifier check in the first version. The missing-source check was the one everyone asked for, and the overstrong paraphrase was the one legal caught.