A convincing AI demonstration can answer a question in seconds. A useful business system must also know whose question it is, which information it can access and what should happen when the answer is uncertain. That difference changes the investment conversation. The unit of value is the completed workflow, including its mistakes, exceptions and maintenance, rather than a fluent response on a screen.
Start where the work gets stuck
Consider a hypothetical service team preparing replies from an internal knowledge base. The model is only one part of the process. Someone must keep the documents current, handle conflicting policies and decide when a customer needs a human response. Faster drafting has limited value if every answer creates a longer verification task somewhere else.
A useful pilot therefore begins with a baseline: time to resolution, rework, escalation and customer outcomes. Teams can compare the assisted process with the existing one using the same kinds of cases. A modest improvement that survives difficult cases is more informative than a striking demonstration built around easy questions.
Give the exceptions an owner
NIST’s AI risk framework provides background on managing risks throughout an AI system’s lifecycle. Our practical interpretation is to give each failure path an accountable owner. Who pauses the system? Who reviews a disputed answer? Who can withdraw access to a sensitive document?
Those questions belong in the operating plan before a rollout expands. They also reveal hidden costs that a price per model request cannot capture. Review time, integration work and incident handling should sit alongside inference spending in the business case.
What progress looks like
The strongest next step is a narrow deployment with a clear stopping rule and a reversible path. Expand when evidence shows that the entire process improves. AI becomes an advantage when people can depend on the work it helps complete.
A hypothetical workflow from request to resolution
Imagine a company that wants an assistant to help staff answer questions about internal procedures. The initial prototype accepts a question, retrieves documents and writes a response. It looks efficient because the demonstration ends when the text appears. The operational workflow continues: the employee must judge whether the answer applies, act on it and deal with any consequences. A serious pilot follows the request through those later stages.
The team can map the journey as a set of decisions. First, establish what the person is asking and whether the request belongs within the system’s scope. Next, identify the information the person is allowed to use. Then retrieve relevant evidence, generate a response and make the supporting context available. Finally, provide a route to clarification, correction or escalation. The model contributes to several stages, but the application owns the relationship between them.
This map exposes where responsibility sits. A document owner may need to resolve a contradictory policy. A service owner may need to define which questions should be escalated. An engineer may need to make an interrupted request recoverable. None of those needs is solved merely by selecting a more capable model. The workflow makes the dependencies visible enough to assign them.
Build the evaluation around decisions, not impressive answers
The pilot’s evaluation set should resemble the distribution of work the organization expects, while including difficult cases with meaningful consequences. A collection consisting only of common, clearly documented questions can show that the prototype works on its easiest path. It cannot establish how the system behaves when the evidence is incomplete or the user’s context changes the answer.
For the hypothetical procedure assistant, the team could include a question with a current source, a question with conflicting sources and a question for which no authorized source exists. It could also test whether a request from a different role produces an appropriate access boundary. The desired behavior may be an answer in one case, a clarifying question in another and a referral in the third. Success needs to be defined separately for those situations.
Keep the original inputs, expected behavior and system configuration together. When a prompt, retrieval setting or model changes, the team can rerun the same cases and investigate any regression. A new version that improves a general impression while weakening an important boundary should not be treated as an unqualified improvement. The result needs interpretation in relation to the workflow’s priorities.
Put the hidden work into the business case
A business case can begin with the time required to draft an answer and overlook the time needed to verify it. If the assisted process moves effort from a junior employee to a scarce specialist, a shorter first response may create a more expensive bottleneck. The evaluation should therefore observe review effort, rework and the amount of unresolved work as well as the speed of generation.
Maintenance deserves a place in the model. The knowledge base changes, the service receives new kinds of questions and the organization revises its procedures. Someone must decide when those changes require another evaluation. A pilot that depends on an enthusiastic individual manually fixing every problem is not yet a dependable operating arrangement. Record that contribution so the company understands the capacity it would need to sustain.
The organization can also compare the AI proposal with a simpler intervention. Better search, clearer documentation or a redesigned form might resolve part of the same problem. This comparison does not assume that conventional software is always preferable. It helps identify which part of the value requires generative AI and which part comes from improving the surrounding information system.
Expand only as the operating evidence improves
A controlled rollout can begin with a defined user group, clear feedback channels and an owner able to pause it. Explain what the tool is intended to do and which decisions remain with people. Users should understand how to report an error and what happens to that report. Collecting feedback without a response process creates the appearance of oversight while leaving the underlying issue untouched.
Expansion can then follow evidence about the complete workflow. The organization may broaden the range of questions, add another user group or connect another information source, but each change should be evaluated for the new dependencies it introduces. The objective is a service that becomes more useful while remaining understandable and governable. That is a stronger foundation for an AI advantage than a sequence of increasingly ambitious demonstrations.
