Home
December 2025

AI is Not the Same Person

Like a Switch

I had an epiphany about AI.

With ChatGPT (or any other AI service), each prompt feels like talking to a single person. We even talk about it that way: "I asked ChatGPT".

There are two ways this is wrong:

  1. AI isn't a person. It's really advanced math and statistics designed to produced responses that sound like a person.
  2. Even if it was a person, it would be a different person each time you prompt it.

It's easy to forget the first point.

The second is subtle enough that I missed it entirely. Now that I see it, I realize it's the core of an uneasy feeling I've had about AI that I couldn't put my finger on.

Personal Knowledge

Spend time with someone and you'll get a good idea of what they know and what they don't.

I know a lot amount programming. Ask me a question on the topic and I'll either know the answer or know where to find it.

My friend Alex knows a decent amount about programming too. He took a couple of courses in college before deciding it wasn't for him. He remembers some things pretty well. Others, not so much.

The AI Shuffle

Ask me a programming question, I'll give you an answer. Ask it again two seconds later, I'll give you the same one.

Do the same thing with an AI, you'll get different results.

For example, here's a couple screenshots where I submitted the same prompt in different windows.

A screenshot of an AI prompt and response screen where the prompt is 'how do I print out binary numbers in python'. The response is a able with four rows of answers.
A similar screenshot with the same prompt and four answer table, but the details of the last answer are different.

Don't worry if you're not familiar with "binary numbers in python". It's a programming thing I'm using to make examples for another project. The specifics don't matter for this post. All you need to know is:

  1. The last answer in each of the "Code Example" columns is different.
  2. The answer in the second one doesn't work.

There's No They're There

While prompting an AI feels like talking to the same person every time, it's better to think of it like calling a customer support line. One where Alex and I are both working. You're in great shape if you get me. Get Alex, and 25% of the answers you get are gonna be bullshit.

The AI equivalent of Alex and me is a bunch of different algorithms running behind the scenes. In the same way you never know who you're gonna get when you call customer support, you never know which algorithm you're gonna get when you prompt an AI. It picks one at random and feeds the answer back to you.

For any given question, some algorithms will provide expert answers. Others are going to be more like getting Alex. The difference is, when you're calling a support line you can tell you're getting different folks. Call in a few times and you'll quickly learn to ask for me if you get Alex.

With an AI, you can't tell which algorithm got selected randomly to respond to you. Did you luck out? Did you get the one that lands on the right answer? Or, did you end up with the one that sucks at it's job? It's impossible to tell.

Past Performance and Future Accuracy

And therein lies the issue. If you get a bunch of correct answers from an AI you start to think of it as an expert. But, it's not expertise. You just got lucky a bunch. The set of random algorithms you got worked out. That doesn't mean the next one will. Over time, you can be sure some of them won't.

-a

end of line

Endnotes

The thing I've realized is how much mental energy I burn whenever I prompt an AI. I couldn't describe it until now, but it's the random call center type response that's the issue. It puts me on high alert for having to critically evaluate every answer.

every. single. fucking. time.

The AI services certainly provides answers faster than searching on my own. But, that speed comes with a high cost. Searching on my own is way less draining.

I can search also maintain a flow state when I'm searching on my own. Prompting an AI breaks my flow every time because I feel like I'm going into a stick fight.

"AI" is a super broad term. It covers all types of systems and services. My experience with it is prompting for answers to programming questions. Those answers are easy to evaluate objectively: If you run the code and it works, the answer is right. If you try to run it and it breaks, it's wrong.

That makes determining its quality easier than in other fields where opinion comes into play.