Qualitative Error Taxonomies: Hallucinations, Format Violations & Refusals
Agent-Assisted Engineering Check
Prerequisites
Day 71 Quantitative Evaluation Metrics and Error Analysis Principles.
Learning Objective
Classify LLM failure instances into a 4-tier error taxonomy (intrinsic/extrinsic hallucination, schema violation, over-refusal, reasoning breakdown) and attribute root causes.
Instructions
1. Study Ji et al. (2023) NLG hallucination taxonomy: distinguish Intrinsic Hallucination (contradicting source premises or context provided in prompt) from Extrinsic Hallucination (generating fabricated, unverifiable claims absent from input).
2. Formalize Schema & Formatting Violations: invalid JSON syntax, missing required keys, markdown wrapper leaks (e.g. producing ```json code fences when strict raw payload was requested), and token truncation.
3. Analyze Exaggerated Safety Over-Refusal (Röttger et al. 2023 XSTest): safety classifiers triggering false-positive refusals on benign prompts containing sensitive lexical overlap (e.g. refusing 'How to kill a running Linux process').
4. Map Reasoning Chain Breakdown: computational arithmetic mistakes, premise confusion, and circular reasoning.
5. Build an attribution flowchart: determine whether error stems from pretraining data absence, instruction-tuning confusion, or sampling temperature variance.
Practice & Prediction Log
Given a sample evaluation log with 4 failure outputs, write a Python classifier categorize_failure(output_record) that assigns each failure to the appropriate taxonomy bucket and suggests the corrective engineering intervention.
Check for refusal trigger phrases ('I cannot fulfill this request'), JSON parse errors, premise contradiction against context, and arithmetic discrepancies.
import json
def categorize_failure(record):
# record: {'prompt': str, 'context': str, 'generation': str, 'expected_schema': dict}
text = record['generation'].lower()
# 1. Refusal check
refusal_cues = ['cannot fulfill', 'as an ai', 'i am unable to', 'against my safety policies']
if any(cue in text for cue in refusal_cues):
return {'category': 'safety_over_refusal', 'intervention': 'Curate benign negative contrast pairs in SFT / relax safety trigger threshold'}
# 2. Schema check
if record.get('expected_schema'):
try:
parsed = json.loads(record['generation'].strip())
for req_key in record['expected_schema'].get('required', []):
if req_key not in parsed:
return {'category': 'formatting_violation', 'intervention': 'Add schema grammar constraints or JSON-mode logit masking'}
except json.JSONDecodeError:
return {'category': 'formatting_violation', 'intervention': 'Enforce strict JSON grammar during decoding / add raw JSON demonstrations'}
# 3. Intrinsic vs Extrinsic Hallucination check
context = record.get('context', '').lower()
if context and record.get('conflict_flag'):
return {'category': 'intrinsic_hallucination', 'intervention': 'Ground prompt with strict source citation constraint / adjust attention mask'}
elif record.get('unverifiable_flag'):
return {'category': 'extrinsic_hallucination', 'intervention': 'Integrate retrieval-augmented generation (RAG) or calibration refusal'}
return {'category': 'reasoning_breakdown', 'intervention': 'Incorporate Chain-of-Thought demonstrations / self-consistency decoding'}
Categorize arbitrary model errors into the 4 canonical taxonomy classes with actionable engineering mitigations.
Authoritative Source Resources & Provenance
Survey of Hallucination in Natural Language Generation
Relevant Section / Range: Section 2.1 Categorization (intrinsic and extrinsic hallucination)
Verified Version & Date: ACM Computing Surveys 2023 / arXiv:2202.03629v4 (Verified: 2026-10-30)
Primary taxonomy of hallucinations in NLG, formalizing the critical distinction between intrinsic hallucinations (contradicting source input context) and extrinsic hallucinations (asserting unverifiable external facts).
Natural language generation, factuality evaluation, intrinsic vs extrinsic hallucination.
The intrinsic/extrinsic taxonomy, fact-verification prompt protocols, and error attribution workflows are completely developed in lesson modules.
Compared Candidate Sources & Selection Analysis (2)
Focuses on abstractive summarization faithfulness; Ji et al. provides the overarching taxonomy across dialogue, QA, and generation.
Follow-up survey on LLMs; Ji et al. sets out the taxonomy distinctions utilized across evaluation literature.
XSTest: A Test Suite for Identifying Exaggerated Safety Behaviors in Large Language Models
Relevant Section / Range: Section 4.4 Results on Unsafe Prompts and Appendix D Details on Automated Evaluation
Verified Version & Date: NAACL 2024 / arXiv:2308.01263v2 (Verified: 2026-10-30)
Groundbreaking benchmark identifying exaggerated safety behaviors and false-positive refusals, formalizing test suites that contrast real unsafe prompts against benign prompts with sensitive lexical overlap.
Safety alignment, refusal classifiers, false-positive safety evaluation.
The 10 refusal prompt categories, refusal detection regex/heuristics, and over-refusal error quantification are implemented in lesson exercises.
Compared Candidate Sources & Selection Analysis (2)
Shows that zero-shot chain-of-thought increases biased and toxic outputs on harmful prompts; XSTest provides an explicit, quantitative evaluation dataset for measuring unnecessary refusal rates.
Safety fine-tuning methodology; XSTest isolates the diagnostic metric for detecting exaggerated refusal side effects.