What Happens After Your AI Tool Goes Live: How to Judge a Monthly AI Support Retainer

In a September 2026 survey of 1,636 enterprise technology decision makers, 46.9% said their actual AI spend had run over plan (35.6% moderately, 11.3% substantially), 31.8% said it was about on plan, and 5.6% said it was below.¹ The respondents are global enterprises, the answers are self-reported, and the source is a research firm's press release, so it describes large enterprises, not necessarily a company like yours. It still points at the question that follows a launch: now that the AI is live, who is watching what it costs and what it does?

A monthly AI support retainer is a recurring arrangement that pays for the hours it takes to watch and adjust AI after it goes live, and ours works like a subscription. If you are the owner or CEO, the COO or head of operations, or the CFO or controller, and a provider has put one in front of you, this article is for you. It covers what changes after an AI tool goes live, what a retainer should and should not promise, and six questions you can put to any provider's offer, ours included.

If you would rather talk it through, a free 30-minute Strategy Call with an AI adoption specialist ends with the first two or three workflows worth automating, and it is a good place to bring the six questions.

01. What changes after an AI tool goes live

The ordinary moment. The COO asks why the summary the team trusted in September reads differently in October, and nobody changed anything. The CFO sees an AI line on the bill that was not in the plan. The owner, some months after the launch, is asked by the board who is looking after it now.

Behind all three moments can be the same background: licences were bought and few people used them, or the board asked for an AI plan, and now a tool people rely on is live. A launch ends a project. It does not end the work.

Four things that move. After launch, four things change without anyone deciding to change them:

• Use: who uses the tool, how much, and for what.
• Cost: what each month's use adds up to.
• The model underneath: the AI engine the tool runs on, which its provider updates or retires.
• Your own documents and processes: a template changes, a price list is replaced, an account is renamed.

A fifth is demand, because people who see one workflow work ask for the next, and one workflow that hurts, then the next, is how we approach this work. Sorting the changes this way is a working method, not a standard.

Evidence that the model underneath moves. Researchers tested two widely used model services in March 2023 and again in June 2023, on the same tasks, and found that their behaviour had changed substantially. On one task, telling prime numbers from composite ones, one service's accuracy fell from 84% to 51%. The authors begin by noting that when and how such models are updated is opaque.²

That study is 2023 data, an academic test of public services, with one task shown here, and it is used for one point only: the model under a tool can change without your decision. A model provider's own public page says the same thing in plain terms. It says the provider regularly retires older models, that applications relying on them may need occasional updates, and that customers with active deployments get at least 60 days' notice before a publicly released model is retired.³

Read the notice period as a timetable, not a flaw. Something on the other side of your tool runs on a schedule that is not yours, and that is AI model drift from the outside: nobody on your side changed anything.

What the standards call watching. The US National Institute of Standards and Technology (NIST) calls it post-deployment monitoring: AI monitoring after launch, meaning watching an AI system's behaviour and use after it goes live. Its AI Risk Management Framework says monitoring plans should be implemented, and should include ways of capturing and evaluating input from users and other relevant parties, appeal and override (a way to challenge or reverse what the AI produced), decommissioning, incident response, recovery and change management.

The framework also says a system's behaviour should be monitored while it is in production.⁴ Change management, in plain terms, is how a modification is requested, approved and recorded. The framework is voluntary guidance for organizations, not a rule you must follow.

Who does it in practice. A 2025 survey of 351 people, recruited through two newsletters, industry partners and online ads, and sponsored by a company that sells AI governance software, found that 48% of organizations monitor their production AI systems for accuracy, drift and misuse.⁵ "Drift" there means behaviour that changes over time without a deliberate change by you. People who answer a governance survey recruited this way probably lean toward organizations that already care about the question, so fewer than half may not be a low estimate.

If nobody on your side has this job, the retainer question is really a question about who watches. One pattern behind AI that stalls after the licence is that nobody owns AI inside the company. The owner decides who has the job, the COO owns how the tool is used, and the CFO owns what it costs.

02. Usage and cost: the two numbers that drift first

What the cost picture looks like at large companies. The State of FinOps 2026 report drew on 1,192 respondents: 47% from large enterprises (10,000 or more employees), 33% from enterprises (1,000 to 9,999) and 20% from companies with fewer than 1,000 employees, with 34% in North America. In it, 98% of respondents say they now manage AI spend, up from 63% the year before and 31% the year before that. The top AI cost challenges it lists are visibility into AI costs, allocating them to business units, and determining their value, and one practitioner is quoted as saying, "Is your AI providing value? No one can answer that question yet."⁶

Two cautions apply. The FinOps Foundation sells training and certification, and its respondents are people who manage technology costs for a living, not mid-market owners.

Visibility before optimization. The same report describes teams following the path they used for cloud spending: first gain visibility, then build planning discipline, and only then optimize for value. Read as a buyer, that means that before anyone tunes cost, someone has to be able to say who uses what and what it costs. That is what "watch AI usage and cost" has to mean in any proposal, ours included.

Why cost can run away. OWASP, a volunteer security community, lists unbounded consumption among the risks of AI applications: letting users run excessive, uncontrolled requests can lead to economic losses. Its mitigations include rate limits and quotas (caps on how many requests can be made in a set period) and continuous monitoring of resource use.⁷ Its headline cases are attacks, not ordinary use, so the buyer's lesson is narrower: a limit that someone set and someone watches is the control.

Gartner is reported to forecast AI cost overruns too: at least 50% of generative AI initiatives will exceed their planned budgets by 2028, attributed to poor architectural choices and a lack of operational expertise. That comes from a trade publication's account of a paywalled report.⁸ Whether or not that forecast holds, someone has to be given the job of watching cost.

What it means on your side. Because the work runs on your own AI account, the bill and the usage are yours to see. That is a fact about the setup, not a promise of a report. Permissions and usage limits are set during AI tool rollout, a separate service, and the retainer is where usage and cost are watched.

KPMG's third quarter 2026 survey of 314 US leaders at organizations with US$1 billion or more in revenue found that nearly three-quarters (74%) now include cost reviews in AI approval processes, up from 61% the quarter before.⁹ That measures approval, not what happens afterwards, so the question stays: who watches once the approval is signed? KPMG sells advisory and assurance services, so read it as one firm's survey of large companies. The CFO or controller owns the cost line, and the COO owns who uses what.

03. What ongoing AI support is, as we describe it

The definition. Ongoing AI support is a monthly retainer, which works like a subscription: a set number of hours each month to monitor and tune the agents (the AI workflows we build for you), add new workflows, and watch AI usage and cost. On a client project we monitor AI usage. It is the Run stage of Plan, Build and Run, which our introduction to AI adoption explains.

How it is billed. Projects, such as an assessment, a build or a rollout, are billed on time and materials, which means you pay for the time and costs the work takes. The retainer is separate and optional, so you can stop at the project. A custom AI solution can also be supported on a retainer tailored to the client's needs.

Where it sits and what it runs on. It works on your own AI account, with your data and your licences, and there is no platform of ours to rent. A person stays accountable, and nothing goes out or goes live without someone's sign-off. Someone has to spend hours on the habits that keep AI answers trustworthy, and section 05 shows which ones.

What this article does not say. Rates are not listed on our site. They are shared during calls.

This article sets out no number of hours, no response times, no guarantees and no results, so ask for each of them on the Strategy Call, and hold the answers to the six questions that follow. The same goes for any provider. A retainer you cannot describe in writing is one you cannot compare.

04. What the hours should buy, and what you should be able to see

Six questions to ask an AI consultant or any other provider follow. They are working questions, not a standard, and where a standard backs one it is cited. This section covers the first two, and each question is tagged with who at your company should press for the answer.

1. What do the hours buy? Ask: COO or CFO. Ask for the list in writing. Ours has four items, as section 03 says: monitoring, tuning, adding new workflows, and watching usage and cost. Then ask how hours are counted, what happens to unused hours, and what happens to extra work.

We do not answer those three questions here, and a provider who cannot answer them is one you cannot compare with another. Hours are time, not outcomes, so do not accept an outcome promise in place of a list. NIST's profile for generative AI suggests that contracts with outside AI providers address who is responsible for incidents and for changes to the system over time, and that buyers ask for service level agreements (SLAs), which are written commitments on things like response times and availability.¹⁰ If a provider offers one, ask for it in writing and ask how it is measured.

2. What is reviewed each month, and what do you get to see? Ask: COO, with the CFO for cost. Ask for the specific items: usage by person or workflow, cost against a limit someone set, errors or overrides, changes made, and changes requested. NIST's framework names monitoring in production and ways to capture user input, appeal and override.⁴ Its generative AI profile suggests policies and procedures for continuous monitoring of systems supplied by outside providers once they are in use.¹⁰ If a provider promises a monthly report or a check-in, ask what is in it and who reads it.

A test for any summary you are shown. A working method: take the last three things that went wrong or were changed, and ask which line of the summary would have shown each one. If none would, the summary is decoration. Re-testing after a change is covered in the company knowledge assistant article, and the retainer is one place the hours for that can come from.

05. Who answers, how a change is approved, and how you leave

3. Who is accountable when something goes wrong? Ask: owner or CEO. Ask for a named person on each side. Canada's Voluntary Code of Conduct for advanced generative AI, from September 2023, says that organizations developing and organizations managing these systems both have "important and complementary roles", and lists as an outcome that system use is monitored after deployment, with updates made as needed to address risks that materialize.¹¹ The Canadian Centre for Cyber Security's guidance on managed services, from 2020 and used here only to name the idea, says your organization is the data owner and is legally responsible for data security.¹²

Canada's code is voluntary and says it does not change existing legal obligations, and the Cyber Centre's guide is guidance, not law. NIST's profile also suggests clear assignment of liability and responsibility for incidents, and notification of serious incidents that come from systems supplied by outside providers.¹⁰ Our own line is plain: a person stays accountable, and nothing goes out or goes live without someone's sign-off.

4. How is a change approved? Ask: COO. Ask who may request a change, who approves it, how it is tested, who signs it off, and where it is recorded. NIST lists change management among its post-deployment monitoring elements, and says measurable activities for continual improvement should be built into updates, with regular engagement with interested parties.⁴ Review dates and triggers for your AI policy are the seventh line of the AI policy article, and this is where an approved change gets logged.

A change can also be a response to a problem. Houseblend's NetSuite practice worked through one on a client's month-end close. An automated feed was only mostly right: a refund was posted on the wrong date, the figures were thrown off, and the close was blocked. The response was to design a human control layer around the feed: a trained reviewer on the coded transactions for two to three months, plus a mandatory approval layer, until the automation earns trust.

5. What happens when the model underneath changes? Ask: COO, with your IT lead. This is the point of section 01 turned into a question. Ask who finds out, how quickly, who re-tests the answers, and who pays for those hours. NIST's profile suggests testing and managing rollover and fallback (the backup way of working if the AI is unavailable or retired), which may include manual processing, and decommissioning or retraining models that perform outside defined limits.¹⁰

A provider's notice of at least 60 days before retiring a model, as the lifecycle page above describes, is the kind of date a monitoring habit exists to catch.³ The only answer we give is that monitoring and tuning are in the hours. Two habits cost hours and are described elsewhere:

• Putting an "as of" date on data and on every answer, as in the data readiness checks.
• Logging a known bias in the AI's output where the next person will see it, as in our introduction to AI adoption.

A retainer is one way to buy the hours for those habits.

6. How do you leave, and what do you keep? Ask: owner or CEO, with the CFO. Ask who owns the AI account, the data, the configuration and the written record of what was built. The Cyber Centre's 2020 guidance, used here only to define the terms, says vendor lock-in occurs when it is not financially feasible to port data from one provider to another. It advises considering an exit strategy when entering a service contract, and it lists who retains legal ownership of the data if the contract is dissolved among the questions to ask a provider.¹² Ask about data portability too: can you take your data and your configuration with you, and in what form?

NIST's profile suggests avoiding arbitrary or capricious termination of critical AI services.¹⁰ Our sentence is the one in section 03: the work runs on your own account, with your data and your licences. Notice periods, handover terms and exit costs are not stated here, so ask for them in writing from any provider, including us.

06. Do you need a retainer at all, and how to compare proposals

Three ways to keep AI working.

• An owner inside your company who watches it.
• A retainer.
• Projects as needed.

Each option needs a named person, and the governance survey in section 01 found fewer than half of organizations monitoring their production systems.

A retainer does not replace an internal owner. It supplies hours of work for that owner to direct. It is also optional, and you can stop at the project.

When a retainer fits, and when it does not. This is a reading, not a finding, and a judgement, not a rule. It fits when several workflows are live, changes arrive every month and nobody has the hours. It fits less well when there is one workflow, the documents are stable and a named person inside the company has time. What a change looks like depends on the work: a professional services firm's agents that assemble client material will meet new templates, a contractor's will meet changed specifications and prices, and a finance team's will meet new accounts and reports.

How to compare two proposals. Make a sheet with the six questions as rows and score each provider's answer as written, vague or absent:

1. What the hours buy.
2. What is reviewed each month, and what you get to see.
3. Who is accountable, on both sides.
4. How a change is approved and recorded.
5. What happens when the model underneath changes.
6. How you leave, and what you keep.

Prefer the proposal that states what it will not do. Treat any promise of savings, uptime or response time that does not say how it is measured as a gap, and hold our answers to the same standard.

What "continual improvement" means in a standard. ISO/IEC 42001, the management system standard for AI, specifies requirements for establishing, implementing, maintaining and continually improving an AI management system.¹³ The EU AI Act requires providers of high-risk AI systems (a category the Act defines) to establish and document a post-market monitoring system, meaning watching a product after it is on the market.¹⁴ Both are cited only to show that "someone watches it after launch" is the standard expectation, not a vendor pitch. The ISO text is paywalled, the EU Act covers high-risk systems in the EU, and neither is a rule for you by itself.

Where this fits in the series. This is the seventh article in our AI adoption series. The first, our introduction to AI adoption, explains Plan, Build and Run. The second, is your data ready for AI, gives five checks on the numbers behind an AI answer. The third, which tasks AI drafts, which it runs, and which always need a person, sets the rules for who signs off. The fourth, what a company knowledge assistant is, covers an assistant that answers from your own documents. The fifth, how we run weekly client status reports, walks through a report that reads two company systems and is sent by a person. The sixth, why we connect AI to company systems read-only first, covers the staged ladder of access.

Where to read next. How to test an AI partner's "we use it ourselves" claim shows how to check what a provider says it runs on itself.

This content may be AI-generated and may contain inaccuracies. It is not professional, legal, or financial advice. Verify independently before relying on it.