A distribution network manager I worked with had thirty years of experience in power operations. He knew the network better than anyone. He could tell you which feeders were likely to trip in wet weather, which transformers were running hot, which areas had the highest theft risk. When we deployed a predictive fault model that identified many of the same patterns, he ignored it. Not because the model was wrong. Because the model was right in ways he had not expected, and that made him suspicious rather than confident.

This is the human side of data-driven decision making — the side that technology projects consistently underestimate. The algorithm may be excellent. The analysis may be rigorous. But if the people who need to act on it do not trust it, it produces no value.

Why Experienced People Reject Algorithms

The Explainability Gap

A machine learning model that predicts a transformer failure cannot always explain why it is making that prediction in terms the field engineer recognises. The model has identified a pattern in hundreds of variables across thousands of assets — a pattern that may be real and predictive, but that does not correspond to any mechanism the engineer has ever encountered in thirty years of field experience. When an expert cannot relate a recommendation to their own knowledge, their default assumption is that the model is wrong — not that their knowledge is incomplete.

The resistance to AI recommendations from experienced professionals is not stubbornness or insecurity. It is a rational response to opacity. People who are accountable for the consequences of decisions need to understand the basis for those decisions. Black-box models do not provide that understanding.

The Track Record Problem

Trust is built through demonstrated accuracy over time. A new model, however well-validated in testing, has no track record in the specific operational context it is being deployed in. The experienced professional has a mental model of their domain built over decades. They need to see the new model perform well consistently — not just in a pilot, but in the messy reality of daily operations — before they will trust it enough to act on its recommendations.

The Accountability Asymmetry

When an experienced professional makes a decision that goes wrong, they can explain their reasoning, demonstrate that they followed proper procedure, and show that the decision was reasonable given available information. When they act on an AI recommendation that goes wrong, they have less protection. "The algorithm told me to" is not a professionally satisfying defence. Until accountability frameworks catch up with AI deployment, experienced professionals will rationally be more cautious about algorithm-driven decisions than their own judgment.

How to Build Human Trust in AI Systems

Start with the cases where the model and the expert agree. Show the expert the model output alongside their own assessment. When they match, note it. When they differ, investigate together. This process does two things: it validates the model in the expert's frame of reference, and it identifies the cases where the model may be detecting patterns the expert has not seen.

Make the model's reasoning visible wherever possible. Show not just the prediction but the top factors driving it — and express those factors in operational language the expert recognises. The transformer has elevated oil temperature, above-average load history, and a fault in the same feeder section six months ago. The expert recognises these signals. The model becomes a colleague rather than a black box.

Finally, give experts the ability to override — and learn from those overrides. When an expert overrides a model recommendation and is right, that information improves the model. When they override and are wrong, that builds evidence for trust. The override mechanism is not a concession to resistance. It is a critical part of building the human-AI collaboration that makes both the expert and the model more effective over time.

Work With Dr. Sunny Okonkwo

Ready to deploy AI that actually changes how your organisation operates? Let's talk.

📅 Book a Free Discovery Call View Consulting Packages
Dr. Sunny Okonkwo

Dr. Sunny Okonkwo

AI Strategist · Decision Intelligence Expert · Digital Transformation Leader. Head of Data Analytics at one of Africa's largest energy and utility companies. Author of 7 books including the #1 International Bestseller The AI Alchemist. Keynote speaker at IIBA, Big Data Summit Canada, Global Summit, and UNICAF.