When you ask AI a question, AI does not look up the answer. AI predicts what a good answer would look like based on patterns from training data. If those patterns point toward the correct answer, great. If they point toward something made up, AI writes that with the same confidence. AI cannot tell the difference.
This is why AI can write a perfectly structured paragraph about a historical event and get the dates completely wrong. The structure looks right because AI learned how paragraphs work. The facts are wrong because AI never learned what is true. For AI, a real fact and a made up fact feel exactly the same. Both are just the next most likely word.