The AI adoption metric most IT teams aren't tracking

Picture a mid-sized firm that switched on an AI assistant for 400 staff in February. By May, the dashboard shows a bit over a third of licences in regular use. The vendor suggests more training. The CFO wants to know why the business is paying for hundreds of seats nobody opens. And the IT team, who got everything right on the technical side, can't really explain what's going on.

Plenty of CIOs will recognise some version of this.

The usual explanation is "change resistance", which is a label more than a diagnosis. Maybe the tool is clumsy for the kind of work the finance team does. Maybe people had a one-hour demo in week one and nothing since. Maybe a rumour went around that the rollout is the first step towards a restructure. It could be all three, and the dashboard can't tell you which.

You find out by asking. That sounds obvious, but in most rollouts the only structured feedback channel is the ticket queue, which records what broke and nothing about what people think. Running employee engagement and experience surveys at set points during a rollout gives you the reasons behind the numbers, which is the part you need if you want to change them.

Ask about what actually stalls adoption

A generic satisfaction question won't get you far. If you ask people to rate the new tool out of 10, you'll learn they're unhappy and be left guessing about the cause. The useful questions go after specific blockers.

Start with trust. Do people believe what the tool produces, and do they know when to double-check it? Someone who got a confidently wrong answer in week two may have quietly given up, and you'd never know from the usage figures.

Then purpose. If leadership hasn't said plainly what the tool is for and what it means for people's roles, staff will draw their own conclusions, and they're rarely generous ones.

Workload is worth asking about too. Early on, checking AI output can make work slower, and staff will tolerate that for a while if they can see it's temporary and someone has noticed.

Finally, find out whether people feel they can say "I don't understand this" without it counting against them. In teams where that's awkward, confusion goes underground and bad habits spread.

Survey three times, and keep it short

Run a baseline before the change is announced, so you know what attitudes looked like beforehand. Run a short pulse survey a few weeks after go-live, when complaints are fresh and usually cheap to fix. Then check again around four to six months in, once the novelty has worn off and you can see whether the tool has actually stuck.

The pulse surveys should be brief. Five or six questions is plenty. People in the middle of learning a new system won't thank you for a 40-question form, and the ones who do fill it in probably won't be representative.

Make it safe to be honest

Very few people will tell their manager they're worried an AI tool is going to cost them their job. They'll say it's going fine.

If you want real answers, the survey has to be anonymous, and ideally run by someone outside the reporting line of the teams being asked. Results should be reported in groups large enough that nobody can be singled out. If staff suspect their answers can be traced back to them, you'll get polite responses and very little you can use.

Then do something with what you hear

Go back to that firm with the unused licences. Suppose the survey shows the finance team avoids the assistant because nobody has told them whether they're allowed to paste client data into it. That's a policy gap, and a one-page guideline will fix it faster than another training session.

Or suppose customer service reports that the tool adds work, because every draft reply needs rewriting before it can go out. That points to a configuration or workflow problem, and it sits squarely with IT.

Neither issue would show up on a usage dashboard. Both are straightforward to fix once you know about them.

Whatever you find, tell people what came out of the survey and what's changing as a result. If staff take the time to answer and then see nothing happen, fewer will respond next time, and the ones who do will be less candid.

This works best when IT and HR run it together. IT knows the system and what can realistically be changed. HR knows how to read workforce sentiment and how to communicate awkward findings. Line managers usually hear the grumbles first, so they need to be in the loop as well.

Where to start

Boards are asking harder questions about what their AI spending is delivering, and usage numbers are only part of the answer. If your adoption curve has flattened and nobody can say why, that's the cue to go and ask the people who are meant to be using it.