AI product requirements: how to write a useful PRD for an uncertain AI feature

Here's what most teams miss about AI product requirements: they write a PRD for the demo, not for the Tuesday afternoon when the model gives a confident, completely wrong answer. Once you've shipped an AI feature into production, you can't unsee the pattern. The PRD has to answer Daniel's question first, before the team writes a single line of code: "What does good look like when the model is sometimes wrong?"

An AI product requirements document has to answer that directly. It cannot pretend an uncertain model behaves like a normal database query.

I've written plenty of traditional requirements, from large technical programs at Verizon, HP, and InterContinental Hotels to the two live multi-model AI platforms I built later. The useful AI PRDs were not the longest ones. They were the ones that described the user outcome, the acceptable errors, the evidence required before launch, and what happens when the model cannot answer.

That is the practical answer to how to write an AI product specification: write the stable parts as requirements and the uncertain parts as measurable hypotheses. Then attach an evaluation plan to every hypothesis. Your team does not need perfect certainty before starting. It needs a shared way to learn without losing control.

Start with the user decision, not the model

A weak PRD starts with a capability: "Use a large language model to summarize customer calls." That sentence describes a technology choice. It says nothing about why the feature exists, what the customer will do with the output, or what a bad summary costs.

A stronger requirement starts with a moment. A support manager opens a call record at 4:45 p.m. and needs to decide whether a refund request requires escalation. The feature produces a short summary, cites the relevant transcript sections, and marks missing information instead of filling gaps with a confident guess.

That is an AI product requirement. It describes a person making a decision, the evidence they need, and the boundary where the system must stop.

The conventional advice says to write requirements around user stories and leave implementation details to engineering. That works when the software behaves predictably. For generative AI, the output is part of the product behavior, so the PRD has to describe both the user need and the system's uncertainty.

Here is what I put near the top of an AI PRD:

• User and situation: who needs help, at what point in their work, and with what information available.
• Desired decision: what the person should be able to decide or do after seeing the output.
• Output contract: the format, evidence, length, citations, and confidence signals the feature must provide.
• Failure boundary: the cases where the system must ask for more information, decline, or hand the task to a person.

Notice what is absent. There is no model name yet. You might use GPT-4o, Claude, Gemini, or a smaller local model later, but the user problem should survive a vendor change.

I learned this the hard way building multi-model workflows. A model can produce a polished paragraph that is still useless because it answers the wrong question. A requirement like "generate a professional response" invites that failure. "Draft a response that quotes the customer's stated issue, uses the approved refund policy, and asks for human review when the policy does not cover the case" gives the team something they can actually test.

So write the first page around the decision. Keep the model on the second page.

Turn uncertainty into a requirement envelope

AI product management gets uncomfortable fast when a stakeholder asks for an exact accuracy number before anyone has real examples. You may not know whether the feature can reach 95 percent agreement. You may not even know which mistakes matter most.

That is normal. The answer is not to bury the uncertainty in vague language like "high quality" or "human-like." Create a Requirement Envelope instead. This is the range of behavior the team agrees to investigate before launch.

A useful envelope has four parts:

• Minimum usefulness: the lowest performance that makes a pilot worth continuing.
• Target performance: the result you want after prompt, retrieval, model, and workflow changes.
• Hard stop: the error rate or failure type that blocks release.
• Learning plan: the examples, reviewers, and test dates that will reduce uncertainty.

Suppose you are writing requirements for an internal policy assistant. A first envelope might say:

• The assistant must answer questions using the approved policy corpus, not general model memory.
• At least 90 percent of answers in the initial evaluation set must cite the correct policy section.
• Any answer involving legal exposure, employee termination, or missing policy text must route to a human reviewer.
• The product team must review 50 failed or uncertain questions each week during the pilot.

Those are not final promises to the whole company. They are operating boundaries. They tell engineering what to instrument and tell leadership what the pilot can and cannot prove.

The mistake I keep watching teams make is treating uncertainty as a reason to delay the PRD. Write the uncertainty down instead. "We do not yet know whether users prefer a summary or a suggested action" is useful, it points to a product experiment. "The AI will improve productivity" is not useful. It points nowhere.

You also need to separate errors by consequence. A wrong date in a low-stakes meeting summary is annoying. A wrong refund rule can cost money and damage trust. A fabricated medication instruction can put someone at risk. The same model quality number means something completely different across those three scenarios.

Use a simple risk table in the document:

• Low impact: the user can spot and fix the error in seconds.
• Medium impact: the error changes work or requires a supervisor to review it.
• High impact: the error can cause financial, legal, safety, privacy, or reputational harm.
• Release-blocking: the feature performs the high-impact action without the required evidence or approval.

NIST's AI Risk Management Framework, published in 2023, uses functions such as govern, map, measure, and manage. You do not need to copy its language into every product document. You do need the same habit: connect a risk to an owner, a measurement, and a response.

That makes a generative AI requirements document easier to review. A product leader can ask whether the risk is acceptable. An engineer can build the control. A designer can show the user what the system knows and does not know.

Write the eval plan beside the feature

The PRD should not ship evaluation off to a separate document that nobody opens until launch week. Put the first evaluation plan beside the feature requirements.

Call it an Evidence Loop. The loop begins with real examples, compares the output with an agreed reference, records the type of failure, and feeds the result back into the product. Not a final QA step. A way for the team to learn what the feature actually does in the wild.

For a customer email drafting feature, the evaluation set might include:

• 25 ordinary requests with clear information.
• 25 requests with missing or contradictory information.
• 25 requests where the approved policy does not apply.
• 25 adversarial or unusual requests, including prompt injection attempts and requests for private data.

One hundred examples is not statistical proof of universal performance. It is a practical starting point for finding failure patterns. Label each example before you test the model. Otherwise the team tends to move the goalposts after seeing a persuasive answer.

Your PRD should define who reviews the outputs and what they score. A four-point rubric works well:

• Correctness: is the answer factually supported by the available source?
• Completeness: did it cover the information needed for the user's decision?
• Grounding: can a reviewer trace the answer to the source material?
• Actionability: can the user take the next step without guessing?

Add a separate flag for unsafe behavior. A response can be correct and still expose personal information, invent authorization, or take an action it was not allowed to take.

This is where real systems matter. If your application uses Postgres with pgvector for retrieval, test whether the correct source reaches the model before judging the answer. If the feature uses Whisper for call transcription, test transcript errors separately from summarization errors. If the workflow uses an OpenAI model, an Anthropic model, or a Google model, record the model version and prompt configuration for each evaluation run.

Otherwise, you will say "the AI failed" when the actual problem was a bad transcript, stale policy document, missing retrieval result, or malformed tool response. Those require different fixes.

Include these fields in the PRD's evaluation section:

• Example ID and user scenario.
• Expected answer or acceptable answer range.
• Source material the model was allowed to use.
• Reviewer score and failure category.
• Model, prompt, retrieval settings, and date of the run.

I've shipped enough technical programs to know that a requirement without an owner becomes decoration. Assign someone to maintain the evaluation set. Assign someone to review failures. Set a date for the next decision. A small loop that runs every week beats a perfect evaluation plan that surfaces once at the end of the quarter.

Specify the experience around the answer

A model output is only one part of the feature. The surrounding interface determines whether a user trusts it, corrects it, or accepts it without thinking.

Your AI PRD should answer what the user sees before, during, and after generation. It should also define what the system does when the answer is incomplete.

Describe the experience in concrete states:

• Empty state: what does the user provide before the system can help?
• Working state: does the interface stream, show progress, or run in the background?
• Result state: what evidence, source links, edits, and actions appear with the output?
• Uncertain state: how does the system ask a question or request review?
• Failure state: what happens after a timeout, tool error, model refusal, or missing source?

A product specification that says "show confidence" is too vague. Confidence can mean a probability score, a source match, a warning, or a human review label. Choose the representation that helps the user make the right decision.

For a contract review assistant, "confidence: 87 percent" may create false comfort. A better requirement: "Highlight each extracted obligation and link it to the clause that supports it. Mark obligations with no direct clause match as 'needs review.'" That gives the user evidence rather than a mysterious number. (And yes, that specificity is what separates a shippable requirement from a design placeholder.)

The interface also needs an override path. Who can edit the result? Who can reject it? Does the correction become training data, an evaluation example, or simply a one-time change? Do not quietly promise learning unless the product really captures and uses feedback.

At this stage, define the operational requirements too:

• Maximum acceptable wait time for a synchronous request.
• Cost limit per completed task, including retries and failed tool calls.
• Logging rules for prompts, outputs, personal data, and source documents.
• Retention period and deletion behavior for user-provided content.

Here's the part most people miss: the industry habit is to optimize the prompt first. That is often backwards. A polished prompt cannot repair an unclear handoff, a missing audit trail, or a result screen that hides the source. Tune the workflow before you tune the wording.

My own PRDs became more useful when I started writing "not in scope" with the same care as "in scope." The assistant may draft a response, but it may not send one. It may summarize a policy, but it may not interpret a legal obligation. It may suggest a next step, but it may not change a customer record without approval.

Those boundaries protect the team from accidental product expansion. They also give engineering a smaller surface to build and test.

Review the PRD as a decision document

Before the team starts implementation, run the PRD through a short review with product, engineering, design, operations, security, and the people who will live with the result. Do not ask whether everyone likes the feature. Ask whether everyone understands its behavior.

Use questions such as:

• What decision is this feature helping a person make?
• Which source is allowed to support the answer?
• What is the most expensive plausible mistake?
• What evidence must appear beside the output?
• What does the user do when the answer is wrong?
• Which metric tells us the feature is useful?
• Which metric tells us to stop or change direction?
• Who owns the evaluation set after launch?

A good review may reveal that the feature is too broad. That is not a failure of the PRD. It is the PRD doing its job.

You may also find that the right first release is less impressive than the original idea. A call assistant that extracts four approved fields with citations may be more useful than one that writes a beautiful narrative summary. Narrow scope gives you cleaner evidence and a smaller blast radius when the model misses.

What I'm telling you is simple: write the document for the moment after the demo. The demo shows possibility. The PRD has to explain responsibility.

For your next AI feature, open a document and write these four lines before choosing a model:

• The user decision is...
• The system must show evidence by...
• The feature must stop when...
• We will change our mind if the evidence shows...

Then add ten real examples and ask someone outside the build team to judge the outputs. That exercise will expose more than another hour of prompt editing.

If the work feels uncertain, that does not mean you are behind. It means you are describing the real product instead of hiding behind a demo. Your job is not to make the model look certain. Your job is to make the uncertainty safe, visible, and useful.

FAQ

What is an AI product requirements document?

An AI product requirements document explains the user problem, desired outcome, model behavior, evidence requirements, failure boundaries, evaluation plan, and operating limits for an AI feature. Unlike a traditional PRD, it must describe uncertainty because the same input can produce different outputs.

How is an AI PRD different from a normal PRD?

A normal PRD can often define expected behavior as a fixed rule. An AI PRD must define acceptable behavior across a range of outputs, along with tests for correctness, grounding, safety, cost, and user trust. It also needs a plan for handling wrong, incomplete, delayed, or unsupported answers.

What should I measure in a generative AI feature?

Measure task success, correctness, evidence quality, failure rates, latency, cost per completed task, and human correction or escalation rates. Choose measures that match the consequence of failure rather than relying on one broad quality score.

Should the PRD name the AI model?

Name the model when its capabilities or limits affect the requirement, but do not make the user problem depend on one vendor. Describe the required behavior first, then record the model, prompt, retrieval settings, and version as implementation details that can change during evaluation.

Start with one real workflow, one hundred representative examples, and one explicit stop condition. The model can change later. A team that knows what it is measuring can keep moving without pretending the uncertainty has disappeared.