The problem
The client sells project management software to consulting and engineering firms. Their customers told them the same thing repeatedly: responding to a large RFP takes a senior person several days, and most of that time is spent finding and adapting answers the firm has already written in past proposals. The client wanted a new product line that produced that first draft automatically, built as a real product with billing, permissions, and auditability, not a chat window bolted onto their existing app.
What we actually did
A data model built around sources, not documents
We treated every past proposal as a set of reusable answer units: a question, the answer text, the proposal it came from, whether that proposal won, and when it was written. Uploaded PDFs and Word files were parsed into these units with a structure-aware parser that kept headings and tables intact, then embedded into a per-customer vector index in PostgreSQL with pgvector. Each tenant's data was isolated with PostgreSQL row-level security, so retrieval could never cross customer boundaries.
Structured generation with mandatory citations
When a user uploads a new RFP, the system extracts its questions, retrieves the top candidate answer units for each, and asks the model to draft a response using only those sources. The model returns structured output: the draft text plus the ID of every source unit it used. Any draft that cites nothing, or cites a unit that was not in its retrieved context, is rejected and regenerated automatically. In the interface, every sentence links back to the past proposal it came from.
An evaluation harness that gates every change
Before launch we built a test set of 400 real RFP questions with answers reviewed by the client's own proposal writers. Every change to prompts, retrieval settings, or model version runs against this set and is scored on grounding (is every claim supported by a source?), completeness, and unsupported-claim rate. A change that raises the unsupported-claim rate cannot be merged. This is what let the team switch model versions twice during the project without guessing.
Challenges and tradeoffs
- Customers' past proposals contradict each other. Pricing, headcount, and certifications change over time. We added a "facts" layer where each firm maintains current canonical values, and drafts pull those values instead of whatever an old proposal said.
- Streaming versus validation. Users wanted to watch drafts appear, but citation validation needs the complete output. We stream a draft per question and validate each one as it finishes, marking unvalidated text visually until it passes.
- Unit economics had to work at launch. We modeled cost per RFP from real question counts and routed simple, high-confidence questions to a smaller, cheaper model, keeping gross margin inside the client's target for the new tier.
Results
The product launched as a paid add-on twelve weeks after kickoff. In the first quarter, customers' measured time to a reviewable first draft dropped by 68% compared with their own pre-launch time logs for similar RFPs, and every generated answer carried at least one citation back to its source. The client's sales team now uses the per-customer time savings in renewals conversations, which gives them a concrete ROI number instead of a feature list.
What we'd do differently
We would build the canonical facts layer on day one instead of week seven. Contradictory source material is predictable in any product that learns from a customer's own history, and designing for it early is cheaper than retrofitting it.