What to agree on maintaining an AI system or a custom application

Handover isn’t the end

An AI system or application will need changes even when it is well built. The company evolves, a provider updates its service, a model version changes or a failure nobody anticipated appears. If maintenance is agreed in advance, when something happens everyone knows who handles it and with what priority.

I always distinguish between fixing something that doesn’t work as agreed and developing something new. Adapting an integration because the other system has changed sits in between, and it is worth defining how it is handled. The agreement must state what is included and how anything else is priced.

Explore the idea↗

Who takes care of changes

Assign responsibilities before an incident happens.

1. 01Detect
2. 02Resolve
3. 03Check recovery

What have you already defined?

Owner for incoming incidentsAgreed priorities and response timesBackups and a restore testInventory of access and external servicesUpdate and rollback procedure

Someone else must be able to continue

Document who administers the domain, the infrastructure, the code and each provider’s accounts. Your company must have access to everything it owns, with enough documentation for another professional to continue the work if needed. It is good practice, not mistrust.

Also agree on the incident reporting channel and who receives alerts. A good alert states which operation failed, when and how to reproduce it, without including credentials or unnecessary personal data.

Silent failures

The application responding doesn’t mean everything works. If an integrated system changes its API, the interface can stay up while operations stop being recorded without anyone noticing. AI systems carry another risk: a model change can alter answer quality without producing any technical error.

That is why maintenance includes checks on users’ real journeys and a set of reference cases rerun whenever the model or instructions change, plus a procedure to recover failed operations without creating duplicates.

What I would put in writing

I would document how the system is updated and how backups are made. A backup that has never been restored offers no guarantee; you should know what it includes and how much information could be lost in the worst case.

Hours and response times depend on each business and are agreed. I manage enhancements separately from incidents, so one doesn’t crowd out the other.

If you have a system in production and want to organise its maintenance, here is how I work, or book a meeting and we’ll look at your case.