By Joseph Batrin Jr.
A while back I had AI score a business audit for a client. Seven categories, ten points each, seventy points possible. The report came back with a headline number at the top: thirty one out of one hundred. Clean, confident, official looking. Then I added up the seven category scores it had listed in its own document. They came to fifteen.
Not thirty one. Fifteen. Out of seventy, not one hundred. The scale was invented too, because nothing in that document was ever scored out of a hundred. The headline number was not calculated from anything. It was produced because a number belonged in that spot, and the tool generated something number shaped to fill it.
The Myth
The myth is simple and almost everyone holds it without noticing. AI runs on a computer. Computers do arithmetic. Arithmetic is the one thing computers never get wrong. So whatever number the AI hands you must be right, because machines do not make math errors.
That reasoning is airtight and it is also completely wrong, because the thing generating that number is not doing arithmetic.
The Reality
AI is not a calculator. It is a prediction engine. It has read an enormous amount of text, and what it does is produce the most probable next piece of writing given everything that came before. That is the whole mechanism. When it writes a business report, it is not computing your score. It is producing text that looks like a business report, and business reports have a score near the top, so a score appears near the top. Sometimes that prediction happens to land on the correct calculation. Other times it produces a number that simply looks like the kind of answer that belongs in that spot.
The unsettling part is that it produces the wrong number with exactly the same confidence it produces the right one. There is no hesitation, no flag, no lower confidence tone. A false statement and a true statement come out of the same machinery and they look identical on the page.
This is also why it invents sources. Ask for a study and it will produce an author, a journal, and a year, because that is what usually follows a request for evidence. It is generating something shaped like a citation. Whether that specific paper exists is a separate question the tool never actually asked itself.
Why This Costs You
Most AI mistakes are not dramatic. They are quiet, and they hide inside otherwise excellent work. Here is what makes that expensive rather than merely annoying. The mistake almost never arrives alone in an obviously broken document. It arrives buried inside work that is otherwise genuinely good. That audit was accurate nearly everywhere. The analysis was sound, the recommendations were right, the writing was clean. One number in the most prominent position on the page was fabricated.
The most dangerous AI output is not the one that is obviously wrong. It is the one that is ninety five percent correct, because nothing about it triggers your suspicion. You skim it, it reads well, and you send it to a client with a fabricated number sitting at the top under your name.
A Practical Example
Watch where the errors cluster and you will see the pattern. They land on specifics. Numbers, dates, names, prices, citations, quantities, percentages. AI is strong on structure, explanation, and language, and weakest exactly where a fact has to be retrieved rather than composed.
That is a useful map. It tells you the reviewing you actually have to do is not rereading the whole document. It is checking the handful of hard specifics inside it.
What Actually Works
Verification has to sit where the number is expensive. That is the entire discipline. You do not review everything with equal intensity, because nobody sustains that and it wastes the speed you bought. You decide in advance which outputs carry a real cost when they are wrong, and those get checked every single time without exception.
A blog draft with an awkward sentence costs you nothing. A proposal with a wrong price costs you the client. Those two things do not get the same review, and the system should say so before the work starts, not after.
The fix on that audit was not to scold the tool. It was to show the arithmetic in the open. Fifteen out of seventy, listed as the sum of the seven scores, printed right there where anyone can add it up themselves. When the math is visible, a fabricated number has nowhere to hide.
Do This Today
Take the last thing AI produced for you that had a number in it. Any number. A total, a percentage, a projection, a count. Add it up yourself from the parts, or look it up independently.
Do that three times and you will stop wondering whether this applies to your business. You will know exactly which of your outputs can never go out unchecked, and that list is the beginning of a real system.