FDE Foundations · Module 6: AI Decisions
Model Limits and Failure Modes
LLM features fail in known ways: wrong facts stated confidently, broken formats, silent degradation on edge inputs. Guardrails are part of the design, not a patch after launch.
11 min reading
Objectives
- List the failure modes of LLM-backed features
- Design guardrails: confidence, validation, human review
- Set expectations with the customer in the brief
Know the modes
- Hallucination: fabricated facts, citations, or fields, delivered fluently.
- Format drift: output that stops matching the agreed schema under unusual inputs.
- Edge-input collapse: performance dropping on rare document types or languages.
- Drift and versioning: provider updates changing behavior between weeks.
- Injection: instructions hidden in customer data redirecting the model.
Guardrail stack
Constrain outputs to a schema and validate every field. Route low-confidence results to human review with a queue and an SLA. Keep retrieval or rules as the source of truth for numbers; the model reads and proposes, deterministic code verifies. Log inputs and outputs for audit. Rate-limit and budget-cap calls so a runaway loop cannot become a bill.
Expectations in writing
The brief states what the feature does not do, the measured accuracy on the evaluation set, and the path for errors: "Extraction proposes line items; totals are always recomputed by rules; flagged items go to the review queue within 5 minutes." Customers accept honest limits; they do not accept surprises discovered in production.
The failure you named in the brief is a design constraint. The failure you did not name is a support incident.
Quick check
An optional 2-3 question self-check. Answers never leave your device, are not stored, and never count toward any assessment.
Exercise
Write the guardrail plan for a fictional LLM feature that drafts customer-reply emails: three failure modes you expect, the guardrail for each, and the brief paragraph stating limits to the customer.
Pass criteria
Failure modes are specific (not generic), each has a concrete guardrail (schema, threshold, review path), and the brief paragraph states limits and error handling in two to three sentences.