微調後垂直領域評測與基準對照增量（Delta）測算
Agent 輔助工程檢查
前置要求
第 68 天 LoRA 訓練收斂成果與第 67 天基準測試數據。
學習目標
在第 67 天完全相同的留出 Prompt 測試集上評估微調後適配器，精確核算相比基準基準的準確率增量（Delta）、格式遵從度提升與成對相對勝率。
操作指引
學習來源：在本活動現有的閱讀與練習時段內，參考 Hugging Face LLM Course: Fine-tuning LLMs 的 Chapter 11.5: Custom Evaluation; Example Evaluation Pipeline，用自己的話解釋一處與本地任務的聯繫。保留現有練習與驗收檢查；若外部資料無法存取，使用課程自寫的解釋。1. 將訓練收斂的 LoRA 適配器安全掛載至凍結基座模型之上。
2. 採用與第 67 天基線測試記錄的完全相同的解碼超參數（如 temperature=0.1, max_new_tokens=256）運行同樣的 50 道留出測試題。
3. 測算微調後多項硬指標：垂直任務準確率（如由基準的 15% 躍升至 75%）、結構化格式遵從率（如由基準的 30% 提升至 96%）。
4. 嚴格計算絕對增量指標：Delta_acc = Acc_finetuned - Acc_baseline。
5. 展開逐題並排對照（Side-by-Side Review）：觀察基座原有的無序閒聊客套話如何被精準、結構化的專業回復徹底替代。
自主實作與記錄
運行 post_eval.py 評測腳本。核算微調適配器相比第 67 天基線的準確率淨增量（+XX%）與格式遵從率淨增量（+YY%），並驗證 delta_acc > 0.30。
計算 delta_acc = post_acc - base_acc 並執行數值斷言。
import json
post_res = evaluate_prompts(finetuned_model, test_prompts)
with open('baseline_results.json') as f:
baseline = json.load(f)
delta_acc = post_res.accuracy - baseline['accuracy']
delta_format = post_res.format_compliance - baseline['format_compliance']
print(f'Accuracy delta: {delta_acc:+.1%}, format compliance delta: {delta_format:+.1%}')
assert delta_acc > 0.30
能準確給出微調成效的實測淨增量（Delta）統計，並基於對照法則論證有效性。
權威來源資源與權利證明
Hugging Face LLM Course: Fine-tuning LLMs
對應章節 / 時間戳: Chapter 11.5: Custom Evaluation; Example Evaluation Pipeline
驗證版本與日期: Public edition checked 2026-11-01; moving pages may change
Sequenced templates, supervised fine-tuning and evaluation teaching with exercises/quizzes. LoRA method and memory claims are checked against the original paper, PEFT and the local parameter ledger, not accepted wholesale from Chapter 11.4.
Python, PyTorch, language-model training and the preceding chat-format lessons.
Use the original localized explanations, exercises and checks already present in this activity; no external account or purchase is required for that fallback.
對比候選資源與選型分析 (2)
Retained for initialization and the low-rank method, not a beginner teaching sequence.
Retained for actual API and implementation defaults; reference documentation is not a complete teaching course.
Training language models to follow instructions with human feedback
對應章節 / 時間戳: Section 3.6 Evaluation and Section 4.1 Results on the API distribution
驗證版本與日期: arXiv:2203.02155v1 (Verified: 2026-10-30)
Foundational InstructGPT paper defining the SFT objective: maximizing log-likelihood of demonstrator completions conditioned on prompts.
Supervised fine-tuning, conditional autoregressive generation.
The conditional likelihood objective and prompt-conditioning derivations are completely formulated in the review.
對比候選資源與選型分析 (2)
Earlier RLHF formulation on summarization; InstructGPT standardizes the prompt-response SFT training phase.
Focuses on massive multi-task scaling; InstructGPT details the exact prompt-conditioned loss setup.
Stanford Alpaca: An Instruction-following LLaMA Model
對應章節 / 時間戳: Preliminary evaluation
驗證版本與日期: Stanford CRFM Release (Verified: 2026-10-30)
Original open recipe demonstrating lightweight instruction fine-tuning of open models, establishing prompt/response templates, batch scheduling, and learning rate ranges.
Supervised fine-tuning, instruction formatting, and LoRA fundamentals.
Complete instruction templates, JSONL formatting scripts, and learning rate schedules are self-contained in curriculum materials.
對比候選資源與選型分析 (2)
Multi-turn ShareGPT tuning recipe; Alpaca provides simpler, single-turn clarity ideal for foundational instruction tuning study.
Commercially permissive crowdsourced dataset; Alpaca established the academic instruction tuning standard.