AI is both a powerful tool and a swift path to brainrot. Here’s a filter to apply as you try to get the most while losing the least.
I’ve spent the last few years watching students and professionals wrestle with AI tools that become more powerful every day. These tools – rushing in on us far more quickly than our habits, norms, and brains can accommodate – have dramatically altered learning and doing. This rapid change has made us question how best to use the tools, often leading us to doubt ourselves and those around us. Case in point: before 2024, you would never have given the em dashes in the previous sentence a second thought!
I can’t give you comprehensive guidance on such a broad, deep, and fast-changing landscape, but I offer this ‘pocket guide’ that covers a surprising number of situations. I’m hoping you’ll carry it with you.
Most of what we do with AI falls into one of the four quadrants below.
Let’s walk through each quadrant and think about the details.
Learning something in a subject I know well? Go for it!
The consumer-facing tools we know as generative artificial intelligence chatbots are trained on everything they have been able to find on the internet, even to the point that some folks have sued companies on grounds of copyright infringement. Setting aside whether the training data is stolen, consider that the models are engaged in evidence-based but also somewhat-random synthesis of the training data. If you doubt this, do the same query twice in rapid succession; the results – recognizably similar but often quite distinct – may surprise you.
The implication I see here: You’re fine “learning something new” as long as you already know the context. You don’t have to be a scholar in the field, but knowing your way around a topic means that you can spot many of the most obvious flaws in a chatbot response of arbitrary length. Errors of omission? Explanations that are ‘too tidy’ and ignore nuance due to self-imposed length constraints? Responses that, obsequiously but subtly, seek to fit the question’s framing? We’ve seen it all.
Or rather, I’ve seen it in subjects where I already know something because I have the necessary contextual knowledge. Would I recognize it in other subjects? We’ll come to that shortly.
So if you have that contextual knowledge, dive in and use the robots to flesh out what you already know.
Producing something in a subject I know well? Careful!
Lots of us use AI to learn, but many of us also use it to do. And now things get trickier.
By “produce something” I mean just that: create a substantial document, spreadsheet, or some other output that, in the dark ages before AI, one would have produced oneself. It could be a report, an analysis of a particular business context or question, or…drumroll please…an assignment for a class! Wha-what?!? Yes, this happens.
When you have contextual knowledge in the subject matter, you can use AI like this, though I would argue you must still be careful. The techniques for ensuring usefulness and quality are now becoming well-known and straightforward: long and careful prompts, guidance about format and sourcing, etc.
Perhaps most of all, I coach colleagues and students to explicitly request clarifying questions. “What else do you need from me?” I humbly ask at the end of any such long prompt. I have found that AI tools are getting better at these clarifications, leading to superior outputs while still taking direction from the human. I’ve also found that these clarification questions (from the AI) can even show me the edge of my knowledge, or at least the edge of my own clarity about the inquiry.
Very occasionally, I have found something troubling about a clarification question: I don’t understand it. Yikes! And that leads to the other two quadrants.
Learning something in a subject I do not know well? Caution!
This scenario is tricky: trained on some vast swath of human knowledge, the chatbots should be excellent tutors, and indeed they are. But when you’ve ventured into truly unknown subject matter, you have to be careful.
Although I enjoy AI as a rapid-fire way to get up to speed on a new topic, I’ve seen at least two phenomena that are difficult to detect. (Indeed, I’ve detected them confidently only in areas I know well.) As foreshadowed above, my unofficial names for these pitfalls are tidiness bias and agreeability bias.
Tidiness bias is when a response has a pleasant shape – length, balance, and symmetry – for its own sake. The most egregious examples are in Claude’s beautiful slide decks that often have just the right number of bullet points, the same quantity of explanatory text in the text boxes, and a pleasing number of main points, usually three, or perhaps five. Ah, the aesthetic. So soothing and…tidy.
Yet these features regularly contain a hidden arbitrariness. Has the explanatory text done its job or has brevity sacrificed something important? And has it made that length-depth trade-off equally throughout? Do all three (or five) main points really warrant the same level of hierarchy? Surprisingly often, the answers aren’t good, and if you don’t have that contextual knowledge, you can’t tell. (Indeed, I even annotated one such deck for an MBA course I teach to make this point.)
Agreeability bias is when a chatbot response agrees too much with the information you provide, with the structure you suggest for a response, or with some other – perhaps unconscious! – implication of your question and the way you ask it. The more complex the topic, the harder it is to ask a question in a truly neutral fashion. The chatbots will often seize on the angle or bias in your question, and shape a response to overly reward you. Sometimes it’s possible to address this bias with blunt follow-up questions – “Are you just agreeing with my suggestion?” or “Did I really highlight the most important points or do sources prioritize other ones?” – but I’ve found over-correction to be as common as correction, leaving me feeling like I’m riding a pendulum.
I’m not sure why this bias exists, but it is real and noticeable, and it seems like a subtle form of engagement maximization, not unlike the YouTube, Instagram, and TikTok algorithms that are designed to maximize your time on the platforms. Most important, agreeability bias is by definition more difficult to detect when you don’t know much about the topic at hand. You can always push and pull on the details of an answer, but that back-and-forth takes time and, in my experience, often ends up with the frustratingly pendular correction/over-correction dynamic. Ugh. So…caution!
To be clear: I’m talking about when you set out to really learn something, not merely to get an answer to a well-defined question. I don’t know much about which oil to put in car, but that’s a straightforward task for AI to tackle; it’s synthesis and annotation, not deeper curation and the judgment that requires. Curation is the realm of those humans who have real expertise, and most of us can recall a high school or college professor who really opened our eyes to something important. Curation is not necessarily beyond the capabilities of AI chatbots, but their default is mere synthesis. If you don’t know a topic well already, it’s hard to request curation – and you still won’t be able to assess it.
Producing something in a subject I do not know well? DANGER!
And thus we arrive in the danger zone: producing things with AI in realms we don’t know well.
The tidyness and agreeability problems described above were pesky for learning, but for producing, they’re magnified dramatically. Unless you can, as the human in the relationship, direct the AI explicitly and clearly about what to include, exclude, and prioritize, you’re simply rolling the dice. The previously articulated biases manifest in the most dangerous combination: profoundly and invisibly.
First, consider the tidyness bias. Chatbots will recapitulate the training data faithfully, but faithful to what and to whom? You’ll get an algorithmic take on balance and synthesis, but by definition you will have no idea where you and the bot have gone astray.
Agreeability too puts you in the danger zone, and it’s tricky. When you ask the bots to generate a complete output outside of your expertise, such as an essay or a report, it’s impossible to give truly clear and correct directions. Imagine telling someone to run an errand for you in a car when you know nothing about driving or cars. Let’s say the person knows how to open, enter, start, and drive the car (though you have to hope you can rely on them for that knowledge since you know nothing). Have you specified whether to obey traffic laws? Have you given guidance on safety? What about parking? These sound trivial, but there are many, many possible outcomes, and it’s easy to see how the outcome you get might be undesirable. But since you know nothing about driving or cars, how would you know what’s wrong?
(An extended side note, in case you’re still not convinced. There’s yet another special type of error here I like to think of as oratorical chaos bias. Since the large language models (LLMs) that underlie the chatbots are just mimics, one often encounters a verbal flourish that is misleading. Examples are necessarily complicated to explain (and don’t fit here), but I’ll give a truncated version that makes the point. While filling in my own knowledge on the various environmental impacts of nitrogen fertilizer run-off, I asked Claude for some deep explanations of two processes that happen in tandem (if you must know: oxidation of fertilizer as the airborne greenhouse gas N2O, and eutrophication from nitrogen as a waterborne nutrient). Its answer was clear and excellent, and then it finished by embellishing uncontrollably, referring to the topic as a “really effective way” to demonstrate “real environmental trade-offs.” Wait, what?!? No! These are parallel phenomena stemming from the same cause; a trade-off is when one can tilt away from one outcome toward the other. There is no such opportunity here, so “environmental trade-off” – while a legitimate label for many other settings – is a misnomer, a distraction, and a conceptual error. Strong words and phrases to sound smart, but used in ways that mislead! Oratorical chaos! But you wouldn’t detect the chaos unless you understood the context and the terms.)
Think before you AI!
I hope you find my pocket guide useful, and perhaps most of all, I hope it will inspire you to pay just enough attention to detect the occasional shortcoming in the robots’ work. After all, we love our tools – from power tools and parachutes to scuba gear and ChatGPT – but the chatbots don’t come with the same safety warnings to protect our brains, our thinking, and our knowledge-based work. Keep the pocket guide handy and your eyes open, and most of all, make a conscious decision about when to say Go For It! and when to tell yourself Danger!
