The Human Touch

Chapter 2

Who’s Driving the Bus?

The wide nets LLMs cast allow us to process infinite sources faster than a human could. Unfortunately, even superpowered AI information retrieval has its kryptonite. Here’s what you need to know about AI’s limitations—and why an expert instructional designer must (and will) remain in the driver’s seat.

Accuracy

Among generative AI’s most public downfalls is its tendency to lie. It’s called a "hallucination," or "…the phenomenon where AI algorithms and deep learning neural networks produce outputs that are not real, do not match any data the algorithm has been trained on, nor any other
identifiable pattern."

AI means no harm; it’s just programmed that way. Its role is to search its vast knowledge base and return the most probable answer to your query. While it’s smart enough to pass the Uniform Bar Exam (UBE), more than anything it wants to please you. That people-pleasing trait is no problem on a standardized test with clear correct and incorrect answers. But when it comes to nuance, AI delivers facts, errors, and half-truths with equal confidence because it thinks that’s
what you want.

That’s why we must understand when to use AI and when AI may fail. For brainstorming, organizing, and drafting, AI is your MVP. But experienced L&D experts must validate LLM-generated research and content or risk publishing outright fabrications—some of which may be dangerous or costly to the organization and its employees.

Additionally, generative AI does not provide reliable citations for its output and often plagiarizes directly from copyrighted text and visual sources. There is no way to tell when or if a result includes protected material. Even LLMs that provide citations don’t differentiate direct quotes from the paraphrased text. So tread carefully, validate LLM output, and do not publish AI drafts without editing
them first.

AI tools have no concept of knowledge, no concept of truth, and no concept of logic. Just like a child who has overheard their parents discussing a topic and remembers what was said, the AI tool will be able to parrot back ideas and concepts. While these responses may imply ’understanding,’ we actually know that the child (or AI) did not comprehend what they were saying.

—The Learning Guild

Efficacy

James Clear, the author of Atomic Habits, tells a surprising story about a college photography class. A professor asks half the class to produce as many photos as possible. The other half must produce one perfect picture. At the end of the term, the "quantity" students consistently demonstrated excellence over the "quality" students. Clear explains, "In the process of creating hundreds of photos, they honed their skills. Meanwhile, the quality group sat around speculating about perfection. In the end, they had little to show for their efforts other than unverified theories and one mediocre photo."

The lesson for instructional designers is that your experience will differentiate generic cut-and-paste material from powerful, effective learning material. AI is a perfect tool for getting started. Still, only a professional who’s put in the reps can say, "I know a

story that will illustrate this concept and make it relatable to our employees," or "This draft doesn’t quite hit the learning objectives we’re trying to achieve."

Using AI shortcuts beyond their capacity may be tempting, especially for time-pressed or resource-poor leaders. That’s a mistake. Learning leaders must remain in place as seasoned guides, ensuring work is instructionally sound, goal-aligned, and richly engaging. Your role is imperative to support learner needs masterfully. AI can produce content quickly, but only you can say
it’s good.

Bias-free

Generative AI is only as good as what it knows. Remember, generative AI first learned what it knows from the internet, and not all internet content is created equal. The clear risk is that unchecked, generative AI will reproduce several kinds of bias.

The first bias is well-documented. Training data may reproduce heteronormative and gender stereotypes, racist stereotypes, and misinformation with unapologetic confidence. While companies behind AI continue working to correct these issues, learning leaders must continually educate and align themselves with their organization’s Diversity, Equity, and Inclusion (DEI) efforts and broader industry and social DEI thought leadership. The instructional design process will fall apart if the work isn’t vetted for psychological safety and inclusion by human professionals trained to do so.

The second bias risk is less nefarious but may be just as harmful. Even with good intentions, lousy training data or programming can lead to unintended consequences. For example, a White House report detailed two worrying studies. In one, an LLM mistakenly interpreted as negative the phrase "I will fight for people with mental illnesses." In another, gender-neutral hiring ads meant to attract all genders to a STEM career accidentally de-prioritized women applicants because the algorithm "optimized cost-efficiency" over gender parity. Targeting male applicants was cheaper, so women saw the ad less frequently. So it’s critical to program data carefully, test continually, and check results to ensure the AI performs as intended.

One final word of warning: publicly available generative AI is not secure. Do not use these tools with proprietary or data-protected information. These programs continually learn from users and your sensitive information could show up in someone else’s results!