How AIs think, how I think — Why AI’s Knowledge Space Feels Personal

Learning how LLMs work has not only deepened my understanding of AI but also helped me shape how I think about my own mind. The way they organize knowledge in a multidimensional space felt oddly familiar—almost like a reflection of how I intuitively connect ideas in my own thoughts.

Multidimensional space

The main concept that really impacted me was how words (or tokens) are embedded in a multidimensional space, and that dimensions and directions in this space have intrinsic semantic meaning. This part of a 3blue1brown on how LLMs work explains this concept very well.¹

The words “embedded in a multidimensional space” sound much more complicated than they really are. It just means that all the concepts an LLM knows are “placed” somewhere in its knowledge space. And that similar things are located closer.

For example: the concepts of “tree,” “flower,” and “plant” are closer to each other than the concepts of “4K UHD Monitor,” “sailing ship,” or “restroom.”

It also means that direction and “distance” in this space are meaningful. Another example: let’s imagine we’re standing in the concept of “man,” and the concept of “woman” is at 10º north and 2km away.

That means that if we go to the concept of “brother” and walk 2km in the direction of 10º north from our location, we’ll get to the concept of “sister.”

Of course, I’m giving a 2D example, in the video it’s illustrated as a 3D example, and in practice, it’s a multidimensional space. That means that there are lots of dimensions (thousands) and that each carries a semantic meaning. By navigating this space within this many dimensions combined, we can start to introduce nuance and place many more concepts in relation to each other more easily. This example is just a simplification, but it illustrates how meaning isn’t random—it’s structured in ways that can be navigated.

Okay, so why has this idea been so significant to me? Because it gave me a more clear picture of a mental model I’ve been referencing and getting glimpses of for years about how I myself think. Many times in my life, I’ve mentioned two distinct, seemingly unrelated, concepts to someone, justifying that “it occupies a similar mental space in my mind.”

Or that Person A reminds me of Person B because, despite them being very different, I’ve met them in a similar way. So in my mind, their embeddings occupy either a close space or the vectors in some specific dimension bring me from one idea to another.

I’ve heard before that neural networks, despite the name, don’t carry many similarities to a human brain and the concept is a useful metaphor for machine learning but not necessarily a good analogy to humans. I don’t know if that’s true, or if it applies to this example. But this concept of embeddings in a multidimensional space feels very accurate in my mind.

(One big difference between LLMs and humans is how they “update” this knowledge space. While humans are constantly learning and evolving, an AI model’s training is a discrete event. They don’t learn or improve during a conversation—the best a model can do is store useful user information in a database and reference it when needed. The “P” in GPT stands for “Pre-trained.”)

Realizing the significance of this multidimensional space concept has given me a new perspective on how I relate ideas, memories, and even people. Maybe LLMs and human thought aren’t so different after all—or at least, they share a fascinating common ground in how meaning is structured.

That’s one of the reasons I believe LLMs—while not sentient or conscious—are still engaging in a form of real thinking. The Digital Duck Test: When Does Mimicking Thought Become Thinking?

For a deeper dive into how LLMs think, here’s a great related article by Anthropic on this topic.

Written by a human, edited with the help of AI.