LLM Bias - Always or Never

LLMs have two opposite behaviors regarding biasing their output. They’re either very easy to bias, or impossible to do so.

When working with people we need to be careful not to bias our discussions in ways that are hard to undo. That’s especially true when dealing with difficult topics that might evoke strong feelings or when brainstorming and creating new ideas.

How we first approach a subject can unconsciously nudge our (and others’) thinking. That’s why I usually prefer silent and private brainstorming before sharing our ideas with others. That way we’re less likely to direct how the group will think and frame the subject.

That’s how our minds work, it’s reality that we, as creative beings, have to be careful about and work around.

LLMs are different. Large language models are stateless machines, non-learning and unchanging after release. Of course, these models are biased, by definition, during training. They’re sensitive to their training corpus, post-training reinforcement, guardrails, and other techniques labs apply to produce this type of AI. But after release they’re (at least mostly) an artifact frozen in time.

Within a single interaction, what companies usually label as chats, models are strongly influenced by the tokens in their context window. Even the remaining tokens available in their context window can influence the quality of the model’s response. But by clearing the context we get a blank slate to explore completely different directions, without the model being affected by previous messages.

That’s a significant difference from working with humans that we need to take into account and use to our advantage. We don’t need to be overly careful about anchoring or biasing Claude or ChatGPT. In fact, it can be helpful to lean into a direction so we can fully explore it—while always having the option to clear everything and start fresh, exploring entirely different areas of the map of reality without being burdened by our previous work.

That’s also why I’ve come to agree with Mike Caulfield’s approach of just “get it in” first. Supply your raw material to an LLM without steering it. This gives it freedom to explore paths and open possibilities that you might never have considered. After that initial exploration, we can focus on the areas that we’re more interested in.

This knowledge has been implicitly or explicitly used by most people with some AI chatbot experience — starting a new chat when the model starts to behave badly (I’ve been doing it for a long time).

But that explicit realization — that models are mostly unaffected by bias or anchoring after training, and that this is a major difference in how humans work — is something that I have only just put into words. Social habits have me thinking very carefully about how to introduce a new idea and avoid it either being killed too early or irrevocably steered in a certain direction by priming it the wrong way.