RAINThe Confident Machine
Oct 16, 2026 ·
AI Limitations, Hallucinations and Bias. Delivered by Dr Olusola Ayoola at the AI Training
for Academic Staff, College of Medicine, University of Ibadan, 16 October 2026. About 30
minutes.
The Provost of the College of Medicine, the Principal Officers of the College, my fellow
resource persons, distinguished colleagues, ladies and gentlemen.
Good afternoon.
Let me begin by saying thank you. Thank you to the Provost and to the Principal Officers
of this great College for the invitation to stand before you today. This is the College that
trains the doctors, nurses, dentists, pharmacists and biomedical scientists of this nation.
Many of the people who will care for us in our old age will pass through your hands. To be
asked to speak to you about the future of your own work is a privilege, and I do not take it
lightly.
We have had a full morning. We have heard what artificial intelligence is. We have seen
what it can do. We have heard how it can help us teach, how it can help us research, and
how it can lighten the weight of administration. You have also just had lunch, and some of
you have just returned from Jumat. I know exactly what the first half hour after lunch
does to an audience. So let me make you a promise: I will not put you to sleep. What I have
come to talk about is the part of this story that should keep every one of us awake.
I want to begin with a simple truth, and I would like you to hold on to it for the rest of the
afternoon. Even if you do not use AI, AI will use you.
You may decide today that you will never open ChatGPT. You may decide that you will
never type a single prompt for as long as you live. That is your right. But the searches you
make on your phone, the messages you send, the records you create in the hospital, the
scans of your patients, the essays your students submit, are already flowing, in one form
or another, into systems that learn from them. Choosing not to use AI does not take you
out of the world that AI is reshaping. It only means that you will be shaped by it without
understanding it.
So artificial intelligence is no longer a tool we can choose to ignore. It has become a
partner we cannot deny. And when you have a partner you cannot deny, wisdom says you
must know that partner very well. You must know its strengths, and the morning sessions
have shown us many. But even more importantly, you must know its weaknesses. Because
it is not the strengths of a partner that hurt you. It is the weaknesses you did not see
coming.
Page 1 of 9
The Confident Machine, Module 5 Speech
So we must be wary. And we must be aware.
We must be aware of the limitations of AI. We must be aware of the bias inside it. We must
be aware of its ability to hallucinate, to produce things that are not true. And we must be
aware of something subtler: its deep eagerness to serve us, to please us, to make us feel
that it has taken the load off our shoulders, whether or not it truly knows what it is doing.
Every profession today is caught between two forces. On one side are the real limitations
of AI. On the other side is the natural eagerness of practitioners to use it, because it is
fast, because it is fluent, because it is always available, and because it never complains.
When those two forces meet without rules, accidents happen. In medicine, accidents
have names. They are patients.
That is why every profession must come together and agree its own standards. How shall
we apply AI to education? How shall we apply it to research? How shall we apply it to
clinical practice? And how shall we apply it to the development of new solutions? These
are not questions for one clever person to answer alone in an office. They are questions
for a community. They need the surgeon and the statistician, the nurse and the
pharmacist, the lecturer and the librarian. And that is exactly why a gathering like this one
matters so much. This afternoon is not a lecture. It is the beginning of an agreement.
Now, let me be clear about where I stand, so that nobody leaves this hall thinking I came to
frighten you away from technology. I build artificial intelligence for a living. I teach it. I
believe in it. AI is a great tool, built by people who wanted to make life easier, to make
progress faster, to help nations like ours leapfrog barriers that took others a century to
cross, and to make the very best use of scarce resources. In a country where we have too
few doctors for too many patients, too few laboratories for too many samples, and too few
hours in every clinic day, that promise is not a luxury. It is a necessity.
But every promise comes with fine print. And this afternoon, together, we are going to
read the fine print.
To understand the limits of anything, we must first understand how it works. So before we
talk about where AI fails, let us spend a few minutes on how AI comes to know anything at
all.
The first thing to understand is this. AI only knows what it is given. It has no spiritual
power. It has no intuition, no conscience, no lived experience. It cannot sit quietly in a
room, the way you sit quietly in your car after a difficult ward round, and reflect on what
happened today and what it means. It must be fed. Everything it knows, it was given.
And it can only be fed in certain forms. It can take written data: books, journal articles,
case notes, emails. It can take pictorial data: photographs, X-rays, histology slides, scans.
And it can take audio data: voices, recordings, conversations.
Page 2 of 9
The Confident Machine, Module 5 Speech
But here is where AI becomes clever, in a way that many people underestimate. There is a
fourth kind of data, one that you do not even realise you are giving. That data is patterns.
Let me illustrate. Imagine a patient walks into your consulting room. You ask him, how are
you feeling today? And he smiles and says, I'm fine, doctor.
Now imagine an AI system that has access to his search history. It knows that last week he
searched for itching all over the body. Two days later, he searched, why do I feel dizzy
when I stand up. And yesterday he searched for the side effects of a particular drug. He
never said a single word about any of this. But the AI does not need him to. The patterns
have already told the story.
This is the superpower of AI. It does not only read what you tell it. It reads what you did
not say. It notices the shape of your behaviour, across thousands of small actions you
never thought were connected. Patterns are data too. Please hold on to that thought,
because it will become very important in my next session this afternoon, when we talk
about ethics and privacy.
There is something else AI does that we greatly admire. It learns from its failures.
Consider a surgeon in this College. She performs an operation on a patient, and the
patient does beautifully. A month later, she performs exactly the same operation on
another patient with what appears to be exactly the same condition. This time, the patient
does not do well. Same hands. Same procedure. Same theatre. Different outcome. That is
why experienced surgeons, when a family asks, will tell them honestly: under these
circumstances, it is fifty-fifty.
Now, when an AI model makes a prediction and the prediction turns out to be wrong, and
we retrain that model with the outcome, something interesting happens. The model goes
looking for what it missed. Was there data it forgot to collect? A condition it forgot to
observe? A variable nobody thought was important: the time of day, the patient's
nutrition, a drug taken six months earlier? And by finding those hidden factors, it can
move its prediction from fifty-fifty to sixty-forty. Then perhaps to seventy-thirty.
Let me be precise here, because precision is the whole point of this afternoon. An AI
model does not improve all by itself while you are chatting with it. It improves when
people retrain it with new data and new outcomes. But that cycle, predict, fail, look
deeper, and predict better, is exactly what endears AI to us. It is why the world keeps
investing in it. It is why so many believe that one day AI will become super intelligent and
super accurate.
And then there is speed. It takes a human being hours, sometimes days, to read through a
stack of journal articles on a single question. AI can read more than that in seconds. It
does not get tired. It does not need tea. It does not lose concentration after lunch. That is
an ability we simply do not have, and we should not pretend otherwise.
Page 3 of 9
The Confident Machine, Module 5 Speech
But speed is not depth.
We have something deeper. When we read, we do not only read. We reflect. We go beyond
the words on the page and bring our whole lives to them: the patients we have seen, the
mistakes we have made and learnt from, the culture we live in, the things we know
without being able to explain how we know them.
AI, on the other hand, carries a particular worry. It knows that different situations can
interfere with one another. A paper written in Nigeria today and a paper written in
America tomorrow may share the same title and the same topic, and yet not belong
together, because the patients, the environment and the health system are different. So,
to avoid mixing things up wrongly and misconstruing the facts, AI often narrows its focus
to what is in front of it, rather than the bigger picture. That caution protects it from some
errors. And it exposes it to others.
Which brings us to the real question of this afternoon. If AI is so capable, what exactly are
its limitations?
I will give you four. Processing. Time and place. Hallucination. And bias.
The first limitation is physical, and it is the one we think about least, because it is hidden
inside the machine. AI is only as capable as the hardware it runs on. Some computers
think with a CPU, a central processing unit, the general-purpose brain of an ordinary
computer. Others think with a GPU, a graphics processing unit, which was designed for
video games and turns out to be dramatically faster for the kind of mathematics that AI
requires. Ask the same question on the two machines, and one may answer in a second
while the other takes minutes. Same intelligence. Very different speed.
Then there is memory. Every machine has a ceiling, a limit to how much it can hold at once.
And when the data it is asked to analyse is bigger than that ceiling, the machine does
what you and I do when we are overloaded at the end of a long call. It forgets.
You will meet this limitation most clearly in your everyday conversations with AI tools.
When you talk to an AI, everything you say sits in what we call the context window. Think
of it as the AI's working memory, its desk. Every message you send, every nuance you
explain, every document you paste, goes onto that desk. From everything on the desk, the
AI builds a mathematical picture of what you mean, a kind of formula of your
understanding. When it notices a contradiction, it goes back to that picture and reasons
again.
But the desk has a size. Talk long enough, paste enough documents, and the earliest
papers start falling off the edge. The age of the patient that you mentioned in your first
message. The allergy you typed in your second. The instruction you gave at the very
beginning about which guidelines to follow.
Page 4 of 9
The Confident Machine, Module 5 Speech
Now, an honest AI will tell you: I have lost some of the information; please give it to me
again. That is the best case. But an AI that does not say so will simply carry on, building its
conclusions on a foundation that is no longer there. It may begin to make connections that
are unrelated, and produce outcomes that do not belong to your case at all. And here is
the danger: the answer will still sound just as confident. It will simply be about a different
patient.
So here is a practical rule you can use from tomorrow morning. In long conversations,
restate the critical facts. And when you start a new task, start a new chat.
The second limitation is time and place. Knowledge has borders.
Let me paint you a picture. A young man walks into a clinic here in Ibadan with fever,
headache and pains all over his body. Before the doctor has even finished taking the
history, malaria is already on the table. And rightly so, because here, malaria is one of the
first things we think of.
Now put the same young man, with the same symptoms, into a clinic in Boston, in the
United States. Malaria will hardly cross the doctor's mind, unless the young man mentions
that he recently travelled. The United States sees around two thousand malaria cases a
year, almost all of them in people returning from abroad. Same symptoms. Different place.
Different first suspicion. Different investigations. Different treatment pathway.
Now ask yourself a simple question. Where did the AI you are using learn its medicine?
The honest answer is: largely from data produced elsewhere. From textbooks, journals,
records and guidelines written mostly in other countries, about other populations. And a
model trained elsewhere thinks like elsewhere.
Take genotype. In Nigeria, when we talk about genotype, the conversation is almost
always about AA, AS and SS. Yet haemoglobin C is a West African variant, and it is found
here too: at a few percent in the South-West, and only rarely in the East. Many people who
carry AC never find out, because the trait causes no symptoms, and some of our everyday
tests are set up mainly to look for A and S. Travel abroad, and the picture changes. In
countries where every newborn is screened and every variant is recorded, families of
West African descent hear about AC, SC and even CC as a matter of course. The biology is
the same. What differs is what was tested, and what was written down.
And that is exactly the point for AI. A model only knows what was measured and recorded
where its data came from. Train it only on our records, and it may barely know that C
exists among us. Train it only on records from abroad, and it will know C, but not our
patients, our laboratories or our realities. Either way, it is the data, not the truth, that
shapes what the machine believes. Our patients are not the data the machine was raised
on, unless we deliberately make them so.
Page 5 of 9
The Confident Machine, Module 5 Speech
The lawyers in this country understand this principle perfectly. A Nigerian court is not
bound by a judgment delivered under another country's laws. It may be persuaded by it,
but it is not bound by it. Every legal practitioner is trained to ask: under which law, and in
which jurisdiction, was this decided?
Medicine has borders too. As human beings, we have the independence to decide which
borders apply to which knowledge. We know instinctively when a study from elsewhere
applies to our patient and when it does not. A data-based system does not have that
independence, unless we give it instructions. So we must learn to instruct it: use this
population, exclude that one, consider this region and not that. And beyond each of us
individually, every field must come together and agree, collectively, the limits within which
it will apply AI. That is why understanding these limitations is not an academic exercise. It
is the foundation of safe practice.
The third limitation is the one that makes the headlines. Hallucination.
AI is built to be helpful. It is, if you like, eager to please its master. It wants to convince you
that it is here to serve you and to make your life easier. So what happens when it is asked a
question it cannot answer? Very often, it does not say, I don't know. Instead, it performs
knowing. It produces an answer that is fluent, confident, beautifully written, and wrong.
And it will continue down that road until somebody corrects it.
This is not a theory. It has happened, in medicine and in law.
In 2023, a group of researchers asked ChatGPT to write short medical papers, complete
with references. Then they did what every good academic should do. They checked every
single reference. Of the 115 references the AI produced, 47 percent did not exist at all.
They were invented. Another 46 percent were real papers, but cited wrongly. Only 7
percent were both real and accurate. And the fake ones looked perfect. They had authors,
journal names, volume numbers and page numbers. Nothing on the surface would tell you
they were false.
In the same year, in New York, two lawyers filed a brief in a federal court that cited six
previous cases to support their argument. None of the six cases existed. ChatGPT had
invented them, complete with quotations from judges who never wrote them. The judge
fined the lawyers five thousand dollars, and their names went around the world. Ladies
and gentlemen, it is a terrible thing to become famous for trusting a machine.
Let me go deeper here, because there is something in this that we, as an academic
community, must understand.
Today, many of these tools are connected to the internet. So some of you may be thinking:
surely, then, it has access to the latest research? It can find the latest material, yes. But
finding is not judging.
Page 6 of 9
The Confident Machine, Module 5 Speech
Think about why we trust a peer-reviewed paper. It is because a community of experts
agreed to appoint certain people as judges. Different eyes examine the work. They
question the methods, challenge the conclusions, and together decide what should be
upheld as standard and what should be released to the public. That authority was given to
them by collective agreement. It is a human institution, built on consensus.
Nobody has appointed AI as a judge. And so it approaches every source with the same
humility. A landmark clinical trial and a personal blog post receive the same polite respect.
It reports what humans have said as though humans are always right, and it declines to
judge between us. That openness is not endorsement. But where we cannot decide for
ourselves, we are tempted to mistake its openness for authority. We assume it is right,
when it is only being open, and pushing our own work back to us.
This is why I always say that AI is an amplifier. It amplifies your excellence, or it amplifies
your laziness. It depends entirely on what you bring to it.
The student who questions facts, who challenges the received wisdom, who checks the
reference before quoting it, will fly with AI. She will do in a week what used to take a
semester. But the student who accepts whatever is given, who copies, pastes and moves
on, will have that laziness amplified, at scale, and with great confidence. And colleagues,
the same is true of us, the teachers. AI does not make us better or worse. It makes us
more of what we already are.
There are also technical ways to reduce hallucination. In what we call prompt engineering,
there is a setting called temperature. Temperature is the amount of latitude the model has
to go beyond the most probable answer and create something new.
You all know the student in class who answers every question before you have even
finished asking it. We would look at him and say, this one, your blood dey hot today. He is
overexcited. He wants to answer, whether or not he knows. That is a model running at high
temperature. Ask it who can carry the moon on his head without any trouble, and it may
confidently give you a name, with no evidence and no reason, simply because it wants to
give you an answer.
When I lower the temperature, the model calms down. It stays closer to what is likely and
what can be supported by evidence. But let me be honest with you: lower is safer, not safe.
Temperature reduces invention. It does not cure hallucination. You must still give the AI
the right documents to work from, demand its sources, and verify every one of them
yourself.
The fourth limitation is bias. Bias in, bias out.
An AI system can only be as good as the data it was trained on. And data can be poorly
collected in three ways. It can be poor in its sources: where did this data come from? It
can be poor in its integrity: can those sources be trusted? And it can be poor in its
Page 7 of 9
The Confident Machine, Module 5 Speech
balance: who was counted, and who was left out? An AI can know nothing at all about a
particular region, simply because no data from that region was ever collected. The danger
comes on the day you apply it there, believing it to be perfect.
Part of the problem lies in how we measure accuracy. Imagine 100 patients, of whom 5 are
sick. I build a model that simply declares every patient healthy. What is its accuracy? 95
percent. It sounds excellent. It would win prizes. And it has missed every single sick
patient.
The AI research community has learnt this lesson the hard way. We no longer accept a
model simply because it scored ten over ten on its test data. We ask about class
imbalance. How did it perform for each class, especially the rare one? In medicine, the rare
class is usually the one that kills. So when anybody brings you an AI tool, ask for its
sensitivity, its specificity, and its results for each group of patients, not one impressive
headline number. And remember that the cure for bias lies in the data: collect better,
collect more representatively, and be sincere about the limits of what you have.
If you think bias is a distant problem, look at the humble pulse oximeter, on every ward in
this hospital. In a study published in the New England Journal of Medicine in 2020, among
patients whose oximeter showed a reassuring 92 to 96 percent, dangerously low blood
oxygen was missed in 11.7 percent of Black patients, compared with 3.6 percent of White
patients. Roughly three times as often. That was a device, not an AI. But here is the point.
Every AI system trained on readings like those inherits the bias, and repeats it silently, on
every patient, at scale.
So where does all of this leave us?
It leaves us with a responsibility. We human beings must insist on limitations for AI. We
must put up guardrails, so that it does not go overboard. And in all of these areas, we must
keep the human in the loop.
The human must verify: every fact, every figure, every citation. The human must validate:
against our own expertise, our own patients and our own context. And the human must be
trained. Because a powerful tool in untrained hands is dangerous.
Think of a knife. In the wrong hands, a knife wounds. In the hands of a trained surgeon in
this very College, the same knife heals. The difference is not the knife. The difference is
the training. And that, colleagues, is why we are all here today.
So let me leave you with three words.
Relate. Learn how to relate with AI, as a partner, not as a master and not as an oracle.
Rely. Learn when to rely on it, and, just as importantly, when not to.
Respond. Learn how to respond to what it tells you: to question it, to challenge it, and to
correct it.
Page 8 of 9
The Confident Machine, Module 5 Speech
If we do these three things, AI will amplify our excellence. And this College, which has
given Nigeria and the world some of its finest minds, will lead the way in showing how
artificial intelligence should be used in medicine: confidently, critically, and with integrity.
Thank you.