微调后垂直领域评测与基准对照增量（Delta）测算
Agent 辅助工程检查
前置要求
第 68 天 LoRA 训练收敛成果与第 67 天基线测试数据。
学习目标
在第 67 天完全相同的留出 Prompt 测试集上评估微调后适配器，精确核算相比基线基准的准确率增量（Delta）、格式遵从度提升与成对相对胜率。
操作指引
学习来源：在本活动现有的阅读与练习时段内，参考 Hugging Face LLM Course: Fine-tuning LLMs 的 Chapter 11.5: Custom Evaluation; Example Evaluation Pipeline，用自己的话解释一处与本地任务的联系。保留现有练习与验收检查；若外部资料无法访问，使用课程自写的解释。1. 将训练收敛的 LoRA 适配器安全挂载至冻结基座模型之上。
2. 采用与第 67 天基线测试记录的完全相同的解码超参数（如 temperature=0.1, max_new_tokens=256）运行同样的 50 道留出测试题。
3. 测算微调后多项硬指标：垂直任务准确率（如由基线的 15% 跃升至 75%）、结构化格式遵从率（如由基线的 30% 提升至 96%）。
4. 严格计算绝对增量指标：Delta_acc = Acc_finetuned - Acc_baseline。
5. 展开逐题并排对照（Side-by-Side Review）：观察基座原有的无序闲聊客套话如何被精准、结构化的专业回复彻底替代。
自主演练与记录
运行 post_eval.py 评测脚本。核算微调适配器相比第 67 天基线的准确率净增量（+XX%）与格式遵从率净增量（+YY%），并验证 delta_acc > 0.30。
计算 delta_acc = post_acc - base_acc 并执行数值断言。
import json
post_res = evaluate_prompts(finetuned_model, test_prompts)
with open('baseline_results.json') as f:
baseline = json.load(f)
delta_acc = post_res.accuracy - baseline['accuracy']
delta_format = post_res.format_compliance - baseline['format_compliance']
print(f'Accuracy delta: {delta_acc:+.1%}, format compliance delta: {delta_format:+.1%}')
assert delta_acc > 0.30
能准确给出微调成效的实测净增量（Delta）统计，并基于对照法则论证有效性。
权威源资源与权利证明
Hugging Face LLM Course: Fine-tuning LLMs
对应章节 / 时间戳: Chapter 11.5: Custom Evaluation; Example Evaluation Pipeline
验证版本与日期: Public edition checked 2026-11-01; moving pages may change
Sequenced templates, supervised fine-tuning and evaluation teaching with exercises/quizzes. LoRA method and memory claims are checked against the original paper, PEFT and the local parameter ledger, not accepted wholesale from Chapter 11.4.
Python, PyTorch, language-model training and the preceding chat-format lessons.
Use the original localized explanations, exercises and checks already present in this activity; no external account or purchase is required for that fallback.
对比候选资源与选型分析 (2)
Retained for initialization and the low-rank method, not a beginner teaching sequence.
Retained for actual API and implementation defaults; reference documentation is not a complete teaching course.
Training language models to follow instructions with human feedback
对应章节 / 时间戳: Section 3.6 Evaluation and Section 4.1 Results on the API distribution
验证版本与日期: arXiv:2203.02155v1 (Verified: 2026-10-30)
Foundational InstructGPT paper defining the SFT objective: maximizing log-likelihood of demonstrator completions conditioned on prompts.
Supervised fine-tuning, conditional autoregressive generation.
The conditional likelihood objective and prompt-conditioning derivations are completely formulated in the review.
对比候选资源与选型分析 (2)
Earlier RLHF formulation on summarization; InstructGPT standardizes the prompt-response SFT training phase.
Focuses on massive multi-task scaling; InstructGPT details the exact prompt-conditioned loss setup.
Stanford Alpaca: An Instruction-following LLaMA Model
对应章节 / 时间戳: Preliminary evaluation
验证版本与日期: Stanford CRFM Release (Verified: 2026-10-30)
Original open recipe demonstrating lightweight instruction fine-tuning of open models, establishing prompt/response templates, batch scheduling, and learning rate ranges.
Supervised fine-tuning, instruction formatting, and LoRA fundamentals.
Complete instruction templates, JSONL formatting scripts, and learning rate schedules are self-contained in curriculum materials.
对比候选资源与选型分析 (2)
Multi-turn ShareGPT tuning recipe; Alpaca provides simpler, single-turn clarity ideal for foundational instruction tuning study.
Commercially permissive crowdsourced dataset; Alpaca established the academic instruction tuning standard.