--- title: "Chapter 03: The Experiments" description: "Understand what each knob does, check that intuition, then sweep rank, steps, dataset size, targets, and scale — building a contact sheet gate by gate." type: lesson order: 3 chapter: "01-lora-eyes-open" --- # Chapter 03: The Experiments ## Objective Build a **contact sheet**: same fixed prompt, every adapter variant labeled. Learn to name underfit vs overfit by eye. Your tutor runs this **one experiment at a time** — not all five overnight without looking. Prerequisite: Chapter 02 baseline (mflux generates, train folder + captions, at least one completed or in-progress baseline run). --- ## Concept: What the knobs actually do You are not hunting a magic hyperparameter combo. You are building **visual vocabulary** for failure modes you will later see in text and tool-call fine-tunes. | Knob | Intuition | |------|-----------| | **Rank** | Capacity of the adapter. Too low → cannot express the subject. Too high + tiny data → memorizes training photos. | | **Steps / epochs** | How long you press the patch in. Too few → underfit. Too many on few images → photocopy of the train set. | | **Dataset size** | More varied images usually help identity; more near-duplicates do not. Capacity × data is a tradeoff. | | **Target modules** | Which layers get LoRA. Attention-only is often enough; “all linear” is more capacity and more ways to overfit. | | **Adapter scale** | At inference (or baked scale), how hard the adapter pushes. Adapters are **dials**, not on/off switches. | ### What you are looking for in images - **Underfit:** “a dog,” not *your* dog; token does little. - **Sweet spot:** recognizably your subject in **new** scenes. - **Overfit:** same pose/background as a training shot; token works only as a photocopy machine. Write **one prediction** before each run (one sentence). After the contact sheet, score your predictions. The goal is calibrated judgment, not a perfect LoRA. --- ## Concept check **Q1.** You train rank 64 for a long time on **5** near-duplicate photos of a dog on the same couch. What do you expect to see when you prompt `TOK dog on the moon`, and why? **Q2.** Why is a **contact sheet** (same prompt, every variant labeled) more useful for learning than a single “best” image you like?
Concept answer key — attempt first ## Answer key (concept) ### Q1 **Model answer:** Strong overfit: the dog (or couch/pose) from the training photos pasted into a weak “moon” scene, or collapse to training frames. High capacity + tiny/narrow data memorizes examples instead of general identity. **Pass criteria:** predicts memorization/overfit; links high rank or long train + few similar images ### Q2 **Model answer:** Side-by-side comparison teaches the *deltas* between knobs; a single pretty image does not build vocabulary for underfit/overfit you will need later on text. **Pass criteria:** comparison / deltas / judgment over one-off aesthetics
--- ## Gate: Fix the evaluation protocol Before sweeping, lock: 1. **One prompt** used for every variant (include your trigger token + a scene **not** in the train set). 2. **One seed** (or a fixed short list of seeds) so noise does not look like a hyperparameter effect. 3. **Output folder layout**, e.g. `~/lora-eyes-open/sweeps/rank-4/`, `.../rank-16/` — each experiment gets its **own** `checkpoint.output_path` so zips do not overwrite. 4. **Baseline reference:** your Chapter 02 adapter (or note if it is still training). 5. **Resume habit:** if a sweep run dies, continue with `mflux-train --resume …/checkpoints/NNNNNNN_checkpoint.zip` (see [Chapter 02](/fine-tune-local-agent/01-lora-eyes-open/02-train-an-adapter/) — checkpoints are full training state; the data folder must still exist). Do not restart with `--config` unless you intend a fresh run. Paste your fixed prompt to your tutor before the first sweep run. --- ## Gate: The sweep (one experiment at a time) Copy your working `train.json` for each variant; change **one** axis only. Train (or load checkpoints), generate with the fixed prompt, save labeled images. | Experiment | Vary | What you're looking for | |---|---|---| | A | rank 4 / 16 / 64 | Where does capacity stop helping? | | B | steps / epochs: short / medium / long | Where does it flip from underfit to memorized? | | C | dataset 5 / 15 / 30 images (from your raw→train pipeline) | The data/capacity tradeoff | | D | attention-only vs broader linear targets | Is attention enough for *your* subject? | | E | adapter scale 0.3 / 0.7 / 1.0 | Adapters are dials, not switches | For E, bake or load a scaled adapter when your toolchain supports it, e.g.: ```bash mflux-save --lora-paths /path/to/lora.safetensors --lora-scales 0.7 ``` (Exact flags follow your installed mflux version — check `--help` if names differ.) **Pacing:** finish experiment A generations and look at them **before** starting B. Your tutor should stop you from launching all jobs blind. --- ## Gate: Build the contact sheet **Deliverable:** one grid image (or a clearly labeled folder of a single index page) — same prompt, every adapter variant labeled. Pin it somewhere. You will refer back when a **text** LoRA in Parts 2–5 behaves strangely and you need to name what you are seeing. ### Exit criteria - You can look at a generation and say “overfit” or “underbaked” without checking loss curves. - You can predict, before running, roughly what rank 64 on 5 images will do. ### The one failure mode that matters here **Spending three days on this.** It is rung one. Get the contact sheet, move on. --- ## Check your understanding **Q3.** Why change only **one** hyperparameter per experiment when building the contact sheet? **Q4.** How does “adapter scale” (experiment E) differ from “rank” (experiment A) in what it is controlling?
Answer key — attempt every question first ## Answer key ### Q3 **Model answer:** So you can attribute the visual change to that knob. Changing many variables at once makes the contact sheet unreadable as science. **Pass criteria:** attribution / isolate variables / interpretability of the sheet ### Q4 **Model answer:** Rank is capacity of the trained adapter (how much it *can* represent). Scale is how strongly that adapter is applied at use time (how hard it pushes), even for a fixed trained file. **Pass criteria:** capacity vs application strength / train-time capacity vs inference dial
--- [← Chapter 02](/fine-tune-local-agent/01-lora-eyes-open/02-train-an-adapter/) · [Next: Part 2 →](/fine-tune-local-agent/02-the-pipeline/)