AI hallucination
Unlike a classic programming bug, which produces an incorrect result in a predictable and reproducible way, hallucinations are unpredictable: the same model may answer a question correctly in one context and hallucinate in another slightly different one.
Why they occur
Language models generate text by predicting the most probable continuation of a sequence. This mechanism produces fluent, coherent text, but it does not guarantee that the content is true: the model optimizes for linguistic plausibility, not for factual accuracy.
Hallucinations are more frequent when:
- The question concerns very specific facts, dates, proper names, or figures.
- The topic is underrepresented in the training data.
- The model is asked to cite specific sources: it may invent references that look real but do not exist.
- The question contains an incorrect premise that the model accepts without question.
Types of hallucination
- Factual: the model states something objectively false.
- Citation: the model invents bibliographic references with author, title, and year that do not exist.
- Code: the model generates code that references nonexistent functions, APIs, or libraries.
- Reasoning: the model reaches an incorrect conclusion through apparently valid steps.
Implications for agile teams
In the context of agile development with AI, Scrum Manager's manual Historias de usuario con IA [User stories with AI, in Spanish] identifies three direct risks:
- In AI-generated user stories: the model may suggest integrations with nonexistent APIs, propose technically unfeasible features, or invent "best practices" not backed by the industry.
- In generated code: references to nonexistent functions, seemingly correct logic that fails in edge cases, or tests that pass without validating actual behavior.
- In documentation and analysis: invented statistics, quotes, and references that look legitimate.
The risk is amplified by two cognitive biases: authority bias (we tend to trust sources that appear authoritative) and automation bias (we place excessive trust in automated systems).
Mitigation
- Grounding: anchoring the model to verified sources of truth (in-house documents, databases, repository code) to reduce its reliance on training memory.
- Human review: human review of generated code and a reinforced DoD are the main detection mechanism.
- AI supervising AI: using a second agent as a reviewer of the first.
- Calibrated skepticism: treating AI output as a proposal that needs verification, not as an authoritative source.
Resources
🏦 AI applied to agile workSkill Arena · Scrum Manager
📄 Scrum Master in teams with AIFree download · Scrum Manager
Common mistake
Assuming that the model's confidence reflects its accuracy. Hallucinations are especially dangerous because the model expresses no uncertainty when it makes things up: the generated text has the same confident tone as when it answers correctly. Fluency and coherence are not indicators of truthfulness. The more specific and verifiable the generated information is, the more necessary independent verification becomes.
See also
Want to advance in agility? You can look for course and exam calls or go at your own pace by becoming a member of the Agile Club. This membership includes exclusive resources, e-learning classrooms, and access to Skill Arena: a space to practice and measure your agile skills at your own pace.