Chapter 05: Train and Check
Run mlx_lm.lora, then pass the three checks: overfit sanity, loss masking, and adapter vs. merged. The loss masking check is the highest-leverage lesson in the course.
TUTOR WITH THEFOCUS.AI
Copy this prompt into Claude, ChatGPT, or any external AI assistant. It points the assistant to the course instructions and links it to your student profile to track your progress and customize observations.
You are not enrolled yet. Enroll to generate a Student ID to track lesson completions and store learning notes.
Chapter 05: Train and Check
Training
mlx_lm.lora \
--model mlx-community/gemma-3-1b-it-4bit \
--train \
--data ./data \
--iters 300 \
--batch-size 4Flags that matter:
--fine-tune-type—lora(default),dora, orfull--num-layers— how many layers receive adapters (default 16). Drop to 8 or 4 under memory pressure.--batch-size— tutorials default to 1 for 16GB machines. You have 64GB. Use 2–4 and cut wall-clock time.--grad-accumulation-steps— raises effective batch size without raising memory--grad-checkpoint— trades compute for memory--report-to wandb— if you want live curves instead of terminal scroll
Adapters land in ./adapters by default.
The Three Checks (do all three, in order)
Check 1 — The overfit sanity check
Train on 10 examples for 200 iterations. Generate on those exact 10 prompts. The model must reproduce the outputs essentially verbatim.
If it cannot memorize ten examples, your formatting, chat template, or loss masking is broken. This takes two minutes and saves three hours. Run it before every new dataset shape, forever.
Check 2 — Loss masking
Confirm you are training only on assistant tokens. In mlx-lm’s chat format this is handled for you; if you move to a custom loop or to mlx-tune (an MLX fine-tuner with an Unsloth-compatible API), it exposes this explicitly as train_on_responses_only and get_chat_template.
Why it matters, stated plainly, because Part 3 lives or dies on it: if you compute loss over tool outputs and file contents the model merely observed, you are teaching it to hallucinate observations instead of going and fetching them. It will invent the contents of files. This is the single highest-leverage line in the entire course.
Check 3 — Adapter vs. merged
Serve with the adapter attached, and serve merged. Confirm identical behavior.
# adapter attached
mlx_lm.generate --model mlx-community/gemma-3-1b-it-4bit \
--adapter-path ./adapters --prompt "What is 2+2?" --max-tokens 200
# merge into standalone weights
mlx_lm.fuse --model mlx-community/gemma-3-1b-it-4bit \
--adapter-path ./adapters --save-path ./models/gemma-json-1bThe Merging Caveat, Worth Internalizing Now
Merging a weak LoRA delta back into a 4-bit base can round the delta away entirely. The safe default dequantizes the base to 16-bit before fusing. If your merged model mysteriously behaves like the base model, this is why. When in doubt, keep the adapter separate and load it at inference.