Day 68: Unattended Adapter Training, Loss Dynamics & Overfitting Watchdogs

01 120 min

Detached LoRA Training Launch and Metric Multiplexing

Agent-Assisted Engineering Check

Prerequisites

Day 67 Preflight verification and Linux process detachment.

Learning Objective

Launch headless adapter fine-tuning in a detached tmux session, streaming real-time metrics to structured telemetry files while separating passive waiting from active study.

Instructions

1. Start training in tmux from the repository root: tmux new-session -d -s lora_train running projects/lora/train.py (smoke mode on CPU; for a real run replace --smoke with --model --device cuda).
2. Stream metrics: tee standard output to lora_training.log; train.py itself appends one JSON line per optimizer step (step, train_loss, lr, elapsed_seconds, mode) to /telemetry.jsonl.
3. Track memory: monitor nvidia-smi memory allocated; observe that VRAM stays virtually constant without fragmentation spikes.
4. Avoid passive study billing: decouple hours of GPU execution from active analytical study minutes.

Practice & Prediction Log

Launch a detached training session. Verify using ps and tail -f that it runs continuously in the background and writes structured loss entries.

Mark as Completed

Independent Mastery Criterion

Deploy persistent detached LoRA training with robust multi-stream telemetry logging.

Authoritative Source Resources & Provenance

linking Copyright Stanford CRFM (blog post; no open license stated)

Stanford Alpaca: An Instruction-following LLaMA Model

Author: Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, Tatsunori B. Hashimoto | Original Source Link ↗

Relevant Section / Range: Training recipe

Verified Version & Date: Stanford CRFM Release (Verified: 2026-10-30)

Selection Rationale:

Original open recipe demonstrating lightweight instruction fine-tuning of open models, establishing prompt/response templates, batch scheduling, and learning rate ranges.

Source Prerequisites:

Supervised fine-tuning, instruction formatting, and LoRA fundamentals.

Safe Fallback (Offline Resilience):

Complete instruction templates, JSONL formatting scripts, and learning rate schedules are self-contained in curriculum materials.

Compared Candidate Sources & Selection Analysis (2)

Vicuna: An Open-Source Chatbot Impressing GPT-4 (LMSYS) ↗ compared_alternative [Open Access]

Multi-turn ShareGPT tuning recipe; Alpaca provides simpler, single-turn clarity ideal for foundational instruction tuning study.

Databricks Dolly 15k Dataset & Training Recipe ↗ compared_alternative [CC-BY-SA 3.0]

Commercially permissive crowdsourced dataset; Alpaca established the academic instruction tuning standard.

linking arXiv non-exclusive license / Open Access

LoRA: Low-Rank Adaptation of Large Language Models

Author: Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen | Original Source Link ↗

Relevant Section / Range: N/A

Verified Version & Date: ICLR 2022 / arXiv:2106.09685v2 (Verified: 2026-10-30)

Selection Rationale:

Primary literature introducing Low-Rank Adaptation, establishing parameter decomposition W = W0 + (alpha/r)*BA, initialization strategies, and zero inference latency merging.

Source Prerequisites:

Matrix factorization, rank, linear layers, and transformer attention.

Safe Fallback (Offline Resilience):

Complete matrix derivations, tensor dimensions, and pure PyTorch LoRALinear modules are implemented directly in lesson code.

Compared Candidate Sources & Selection Analysis (2)

Houlsby et al. Parameter-Efficient Transfer Learning for NLP (Series Adapters) ↗ compared_alternative [Open Access]

Introduces bottleneck adapter layers in series, which introduces unavoidable inference latency overhead; LoRA's parallel formulation merges with zero latency.

Li & Liang: Prefix-Tuning: Optimizing Continuous Prompts for Generation ↗ compared_alternative [Open Access]

Prepends virtual prompt tokens; reduces available context length, unlike LoRA which preserves full sequence capacity.

02 120 min

Loss Telemetry Dynamics: Instruction Memorization vs Generalization

Independent Mastery Check (no AI assistance).

Prerequisites

Day 68 Activity 1 and cross-entropy loss analysis.

Learning Objective

Analyze validation loss curves against training loss to detect the inflection point where adapter adaptation crosses from general learning into verbatim memorization.

Instructions

Study source: within this activity's existing reading/practice time, use Hugging Face LLM Course: Fine-tuning LLMs, Chapter 11.3: Monitoring Training Progress; Warning Signs to Watch For, then explain one connection to the local task in your own words. Keep the local exercise and acceptance checks; use its original explanation if the external source is unavailable. 1. Plot or tabulate step vs train_loss and val_loss.
2. Phase 1 (Initial rapid descent): both train and val loss drop steeply as the adapter aligns with instruction formats and output styles.
3. Phase 2 (Optimal convergence): validation loss hits minimum (e.g. step 400-600).
4. Phase 3 (Overfitting inflection): train loss continues to decrease toward near-zero while validation loss curves upward. This signifies verbatim token memorization, harming out-of-distribution reasoning.

Practice & Prediction Log

Given a sequence of validation losses [2.8, 2.3, 1.9, 1.6, 1.45, 1.44, 1.48, 1.55, 1.63], evaluated every 100 steps starting at step 100 (index i is step 100 * (i + 1)). Identify the step where overfitting began and determine the ideal early-stopping checkpoint.

Mark as Completed

Independent Mastery Criterion

Accurately identify overfitting inflection points on real validation loss curves.

Authoritative Source Resources & Provenance

linking Apache-2.0

Hugging Face LLM Course: Fine-tuning LLMs

Author: Hugging Face course contributors | Original Source Link ↗

Relevant Section / Range: Chapter 11.3: Monitoring Training Progress; Warning Signs to Watch For

Verified Version & Date: Public edition checked 2026-11-01; moving pages may change

Selection Rationale:

Sequenced templates, supervised fine-tuning and evaluation teaching with exercises/quizzes. LoRA method and memory claims are checked against the original paper, PEFT and the local parameter ledger, not accepted wholesale from Chapter 11.4.

Source Prerequisites:

Python, PyTorch, language-model training and the preceding chat-format lessons.

Safe Fallback (Offline Resilience):

Use the original localized explanations, exercises and checks already present in this activity; no external account or purchase is required for that fallback.

Compared Candidate Sources & Selection Analysis (2)

LoRA original paper ↗ compared_alternative [arXiv.org perpetual non-exclusive license / Open Access]

Retained for initialization and the low-rank method, not a beginner teaching sequence.

Hugging Face PEFT ↗ compared_alternative [Apache-2.0]

Retained for actual API and implementation defaults; reference documentation is not a complete teaching course.

linking Open Access

Training language models to follow instructions with human feedback

Author: Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, et al. | Original Source Link ↗

Relevant Section / Range: Appendix C.1 Details of SFT training

Verified Version & Date: arXiv:2203.02155v1 (Verified: 2026-10-30)

Selection Rationale:

Foundational InstructGPT paper defining the SFT objective: maximizing log-likelihood of demonstrator completions conditioned on prompts.

Source Prerequisites:

Supervised fine-tuning, conditional autoregressive generation.

Safe Fallback (Offline Resilience):

The conditional likelihood objective and prompt-conditioning derivations are completely formulated in the review.

Compared Candidate Sources & Selection Analysis (2)

Ziegler et al. Fine-Tuning Language Models from Human Preferences ↗ compared_alternative [Open Access]

Earlier RLHF formulation on summarization; InstructGPT standardizes the prompt-response SFT training phase.

Chung et al. Scaling Instruction-Finetuned Language Models (Flan-PaLM) ↗ compared_alternative [Open Access]

Focuses on massive multi-task scaling; InstructGPT details the exact prompt-conditioned loss setup.

linking Copyright Stanford CRFM (blog post; no open license stated)

Stanford Alpaca: An Instruction-following LLaMA Model

Author: Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, Tatsunori B. Hashimoto | Original Source Link ↗

Relevant Section / Range: N/A

Verified Version & Date: Stanford CRFM Release (Verified: 2026-10-30)

Selection Rationale:

Original open recipe demonstrating lightweight instruction fine-tuning of open models, establishing prompt/response templates, batch scheduling, and learning rate ranges.

Source Prerequisites:

Supervised fine-tuning, instruction formatting, and LoRA fundamentals.

Safe Fallback (Offline Resilience):

Complete instruction templates, JSONL formatting scripts, and learning rate schedules are self-contained in curriculum materials.

Compared Candidate Sources & Selection Analysis (2)

Vicuna: An Open-Source Chatbot Impressing GPT-4 (LMSYS) ↗ compared_alternative [Open Access]

Multi-turn ShareGPT tuning recipe; Alpaca provides simpler, single-turn clarity ideal for foundational instruction tuning study.

Databricks Dolly 15k Dataset & Training Recipe ↗ compared_alternative [CC-BY-SA 3.0]

Commercially permissive crowdsourced dataset; Alpaca established the academic instruction tuning standard.

03 120 min

Periodic Adapter-Only Checkpointing and Fault Isolation

Agent-Assisted Engineering Check

Prerequisites

Day 68 Activities 1-2 and Day 66 adapter serialization.

Learning Objective

Ensure periodic snapshots save exclusively adapter weights (leaving base weights uncopied), verify file sizes < 50MB, and validate atomic persistence.

Instructions

1. Configure periodic saving at regular step intervals (e.g. every 200 steps).
2. Filter the state dict before saving: only save parameters whose names contain 'lora_'.
3. Verify file sizes: confirm that adapter checkpoints are between 10MB and 40MB, never 10+ GB.
4. Apply atomic write patterns with temporary files to avoid corrupted checkpoints upon host interruption.

Practice & Prediction Log

Inspect the checkpoint output directory during training. Verify that 3 rolling adapter snapshots exist, each under 50MB, and that loading them into a fresh model succeeds.

Mark as Completed

Independent Mastery Criterion

Maintain an automated periodic adapter snapshot rotation with strict size bounds and atomic isolation.

Authoritative Source Resources & Provenance

linking arXiv non-exclusive license / Open Access

LoRA: Low-Rank Adaptation of Large Language Models

Author: Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen | Original Source Link ↗

Relevant Section / Range: N/A

Verified Version & Date: ICLR 2022 / arXiv:2106.09685v2 (Verified: 2026-10-30)

Selection Rationale:

Primary literature introducing Low-Rank Adaptation, establishing parameter decomposition W = W0 + (alpha/r)*BA, initialization strategies, and zero inference latency merging.

Source Prerequisites:

Matrix factorization, rank, linear layers, and transformer attention.

Safe Fallback (Offline Resilience):

Complete matrix derivations, tensor dimensions, and pure PyTorch LoRALinear modules are implemented directly in lesson code.

Compared Candidate Sources & Selection Analysis (2)

Houlsby et al. Parameter-Efficient Transfer Learning for NLP (Series Adapters) ↗ compared_alternative [Open Access]

Introduces bottleneck adapter layers in series, which introduces unavoidable inference latency overhead; LoRA's parallel formulation merges with zero latency.

Li & Liang: Prefix-Tuning: Optimizing Continuous Prompts for Generation ↗ compared_alternative [Open Access]

Prepends virtual prompt tokens; reduces available context length, unlike LoRA which preserves full sequence capacity.

linking Apache-2.0

PEFT: State-of-the-art Parameter-Efficient Fine-Tuning Methods

Author: Sourab Mangrulkar, Sylvain Gugger, Lysandre Debut, Younes Belkada, Sayak Paul, Benjamin Bossan | Original Source Link ↗

Relevant Section / Range: N/A

Verified Version & Date: PEFT 0.10.x / GitHub (Verified: 2026-10-30)

Selection Rationale:

Widely adopted open-source library implementing LoRA configuration classes (LoraConfig), get_peft_model, and adapter serialization.

Source Prerequisites:

PyTorch nn.Module, Hugging Face transformers, and LoRA equations.

Safe Fallback (Offline Resilience):

Standalone, zero-dependency PyTorch implementations of LoRALinear layers and save/reload logic are provided in curriculum scripts.

Compared Candidate Sources & Selection Analysis (2)

Microsoft LoRA Original PyTorch Package (loralib) ↗ compared_alternative [MIT]

Original reference code by Hu et al.; PEFT expands this to broader multi-model ecosystem integration.

Lit-GPT LoRA implementation (Lightning AI) ↗ compared_alternative [Apache 2.0]

High-performance clean implementation; compared alongside PEFT for modular code architecture.

04 120 min

Assessment: Real-Time Fine-Tuning Watchdog and Early Stopping Harness

Independent Mastery Check (no AI assistance).

Prerequisites

Day 68 Activities 1-3.

Learning Objective

Construct and verify a production FinetuneWatchdog script that tracks streaming telemetry, monitors validation loss slope, detects divergence/overfitting, and issues graceful termination signals.

Instructions

1. Implement FinetuneWatchdog(patience=3, delta_threshold=0.005).
2. Read validation evaluation events from telemetry.
3. Track best_val_loss and counter: increment counter when current_val_loss > best_val_loss - delta_threshold.
4. When counter >= patience, emit EARLY_STOP_SIGNAL, notify the runner, and record the step of the optimal checkpoint.
5. Also guard against NaN loss: if loss is NaN, emit CRITICAL_ABORT immediately.

Practice & Prediction Log

Run FinetuneWatchdog on a simulated validation loss stream that bottoms out and begins rising. Confirm it halts training at the exact patience boundary and outputs the optimal checkpoint step.

Mark as Completed

Independent Mastery Criterion

Deploy an automated early stopping watchdog achieving 100% test pass on inflection detection.

Authoritative Source Resources & Provenance

linking Copyright Stanford CRFM (blog post; no open license stated)

Stanford Alpaca: An Instruction-following LLaMA Model

Author: Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, Tatsunori B. Hashimoto | Original Source Link ↗

Relevant Section / Range: N/A

Verified Version & Date: Stanford CRFM Release (Verified: 2026-10-30)

Selection Rationale:

Original open recipe demonstrating lightweight instruction fine-tuning of open models, establishing prompt/response templates, batch scheduling, and learning rate ranges.

Source Prerequisites:

Supervised fine-tuning, instruction formatting, and LoRA fundamentals.

Safe Fallback (Offline Resilience):

Complete instruction templates, JSONL formatting scripts, and learning rate schedules are self-contained in curriculum materials.

Compared Candidate Sources & Selection Analysis (2)

Vicuna: An Open-Source Chatbot Impressing GPT-4 (LMSYS) ↗ compared_alternative [Open Access]

Multi-turn ShareGPT tuning recipe; Alpaca provides simpler, single-turn clarity ideal for foundational instruction tuning study.

Databricks Dolly 15k Dataset & Training Recipe ↗ compared_alternative [CC-BY-SA 3.0]

Commercially permissive crowdsourced dataset; Alpaca established the academic instruction tuning standard.

linking arXiv non-exclusive license / Open Access

LoRA: Low-Rank Adaptation of Large Language Models

Author: Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen | Original Source Link ↗

Relevant Section / Range: N/A

Verified Version & Date: ICLR 2022 / arXiv:2106.09685v2 (Verified: 2026-10-30)

Selection Rationale:

Primary literature introducing Low-Rank Adaptation, establishing parameter decomposition W = W0 + (alpha/r)*BA, initialization strategies, and zero inference latency merging.

Source Prerequisites:

Matrix factorization, rank, linear layers, and transformer attention.

Safe Fallback (Offline Resilience):

Complete matrix derivations, tensor dimensions, and pure PyTorch LoRALinear modules are implemented directly in lesson code.

Compared Candidate Sources & Selection Analysis (2)

Houlsby et al. Parameter-Efficient Transfer Learning for NLP (Series Adapters) ↗ compared_alternative [Open Access]

Introduces bottleneck adapter layers in series, which introduces unavoidable inference latency overhead; LoRA's parallel formulation merges with zero latency.

Li & Liang: Prefix-Tuning: Optimizing Continuous Prompts for Generation ↗ compared_alternative [Open Access]

Prepends virtual prompt tokens; reduces available context length, unlike LoRA which preserves full sequence capacity.