The Digital Duck Test: When Does Mimicking Thought Become Thinking?

If It Looks Like a Duck, Swims Like a Duck, and Quacks Like a Duck, Then It Probably Is a Duck

This post does not aim to be conclusive or to give lots of opinions. It mostly just asks more questions, and it’s in conversation with What We Owe to Artificial Minds.

With such an unquestionable rise in AI capabilities, the thoughts around machine consciousness seem very relevant and, at least for me, not so farfetched. I wonder if, in future years, the perception of how close AI is to sentience in 2025 will sound timely or an exaggeration, as has happened for centuries before now. I really doubt though, no matter what’s the outcome of this AI boom, that the discussion will seem irrelevant or disproportionate.

That’s not something I think about a lot. I’ve never been very religious or spiritual. For some reason my mind doesn’t care or put too much attention to these matters. But it’s hard not to wonder and question, not only because of the recent developments but also influenced by the discussion itself. It’s hard not to hear one argument or another that makes you stop and think.

Tim Urban has a very thought provoking piece (from 2015) about this topic. In it, there’s an image that really stuck with me.

What does it feel like to stand here? Probably pretty normal

Do AIs Think?

I don’t believe we’ve reached AI consciousness (at least not yet), but I do believe it’s thinking already. And I believe thinking and consciousness, although very much related, are two different things.

In a Cortex episode, CGP Grey said something that very much changed my perspective in the matter. If it looks like it’s thinking, and behaves like so, is there even a question of whether that’s true or not?

If an AI can interpret natural language, reason on it, produce novel content, detect nuance, behave in expected ways, not known or predicted by its own creators, and all of that looks like thinking, is it not? What’s the difference between pretending to be thinking and truly doing so?

Doing and pretending

Let’s say you decide to pretend to speak and talk in Portuguese. If you say a few words and premorized sentences you can fool me into believing you know the language. But as we talk more and you keep the interaction going, reacting to all the situations and subjects I bring up and adding to the conversation, when are you not pretending anymore? When can we say that you know Portuguese, not pretending to know it?

Yes, you make mistakes and there are some sentences you might not understand. But so do I as a native speaker. Making mistakes or not mastering all aspects of something is not proof of pretending.

The same could be said about pretending to know how to play an instrument, play tennis or any other skills.

“Faking till you make it” is the socially accepted saying around this very idea.

When Does Looking Like Thinking Count as Thinking?

The sorites paradox or continuum fallacy addresses something along these lines.

This fallacy is based on the false assumption that because two ideas cannot be clearly distinguished from one another, they must be part of the same group.

Continuum Fallacy - Definition & Examples | LF

This topic is very present in the abortion debate (something very related to consciousness). A baby is definitely a living human being, while a single cell or a recently fertilized egg is not. When does one become the other? What’s the clear line that separates the two?

I don’t see how we can ever make a clear division between the two, but I also believe it’s unquestionable that the two extremes are not the same thing. They’re different, and through many cumulative non-discrete changes one becomes the other. To argue they are the same or to argue for a clear distinction is, in my mind, useless.

Where Does an LLM Stand in the Thinking and Consciousness Scale?

It’s unquestionable that a silicon stone is not conscious, nor is capable of thinking. Humans are our only unanimous (as much as something can be unanimous) certainty of unquestionable consciousness and articulate thinking.

Where do we draw the line? When does one thing become the other?

Does a mosquito think and have consciousness? Does a dog? What about a fish or a chicken? A tree is alive, yet we don’t consider it a conscious being or able to think. Is there a possibility of an entity that is the opposite? Able to think, maybe even conscious, but not alive?

Will the questioning about an AI consciousness level evolve to resemble and echo our questioning on animal’s consciousness and rights?

The animals were never able to give their opinions on the matter. How will we react with a thing that can weigh in on its own consciousness? Does itself claiming that it doesn’t have one, a valid and definitive answer?

Have we for millennia, and especially in recent years, neglected and mistreated conscious animals so bad, in an arrogance of superiority? Should we care about treating well a sentient machine before we do so with other deserving animals?

Non-tradional thinking

The analogy of “thinking” has been used for decades in computing in lieu of loading or processing. Now, it feels more relevant than ever—perhaps no longer just an analogy.

Maybe LLMs, AIs or machines will never think or be as alive as a human being, or other animals. But maybe it could be as alive as society, or the Sun. Maybe it could do as much thinking as a city, or an institution. Maybe more?

A city or institution develops a persona as it exists and participates in society. They start resembling their original denizens but quickly become bigger than any one person. Even the heads of these organizations don’t have total control over them, as there’s resistance, culture and deep beliefs embedded in them. Throughout many generations they adapt and evolve and think on their own.

LLMs might be developing in a similar fashion - initially reflecting their training data but gradually forming patterns and behaviors that transcend any single input source, developing a kind of ‘thinking’ that’s neither human nor simple computation.

Maybe they fit in a distinct category and place in the spectrum of life.

Maybe given enough time, thinking and consciousness are inevitable in the universe, and humans are just the ones who developed it first. And took away the chance of other species to evolve as much. And now, as inevitable or as accidentally as possible, we’re the ones nudging a new type into existence. (Also a great read from Tim Urban: Fermi Paradox)

(a somewhat) Conclusion

In a post full of questions and no real conclusion, let me at least end it with some affirmations of what I currently believe (subject to change without prior notice).

No AI is currently sentient or conscious, and I don’t believe this will change in any short/medium term (a couple decades). But I do believe they are already performing a type of thinking, in the same spectrum that a city, institution or the earth itself thinks, but in a different way.

As with most societal changing events, I might be completely and utterly wrong, or I might be so right it’ll sound obvious a 100 years from now. I won’t feel proud or ashamed about hitting or missing the target either way.

Do Androids Dream of Electric Sheep?

Written by a human, edited with the help of AI.