Fine-Tune a Self-Improving Local Coding Agent
A five-module ladder from your first LoRA to an agent that trains on its own verified successes. Apple Silicon primary, Jetson Thor for scale.
TUTOR WITH THEFOCUS.AI
Copy this prompt into Claude, ChatGPT, or any external AI assistant. It points the assistant to the course instructions and links it to your student profile to track your progress and customize observations.
You are not enrolled yet. Enroll to generate a Student ID to track lesson completions and store learning notes.
Fine-Tune a Self-Improving Local Coding Agent
A five-module ladder, Apple Silicon primary, Jetson Thor for scale.
5 parts · 16 chapters · Will Schenk
The Constraints (fixed)
- The model is fine-tuned. Not just prompted, not just retrieved-into.
- The model is local. Weights on disk, inference on your hardware.
Everything in this course serves those two. Retrieval, tool design, and context management appear only where they generate training data or gate a deliverable.
What You’ll Build
By the end of this course you’ll have:
- Calibrated intuition for LoRA hyperparameters — rank, alpha, steps, dataset size — built where you can see the failure modes
- A repeatable
data → train → fuse → servepipeline that runs in minutes - An agent harness that logs every trajectory in a trainable format
- A fine-tuned small model that emits valid tool calls where the base model can’t
- A verifier: a binary, mechanical, uncheatable pass/fail scoreboard on real coding tasks
- A flywheel: an agent that attempts tasks, keeps only verified successes, retrains on them, and scores higher each round — with no teacher in the loop
Hardware Split
| Box | Role | Toolchain |
|---|---|---|
| M4 MacBook, 64GB | Iteration, dataset work, all of Parts 1–4 | MLX (mlx_lm.lora, mflux) |
| Jetson AGX Thor, 128GB | Scale runs, Nemotron, final serving | Unsloth / CUDA, Ollama |
Model choice on the Mac: Gemma 3. Nemotron’s hybrid Mamba2-Transformer architecture is still maturing in mlx-lm — it runs for inference but fights you for training. Save it for the Thor. Gemma 3’s ladder (270M → 1B → 4B → 12B → 27B) is exactly the ramp this course wants.
The Spine
Each module produces an artifact the next one consumes. Nothing is decorative.
1. mflux LoRA → intuition about the knobs
2. Gemma 1B toy tune → a working train→fuse→serve pipeline
3. Trajectory logger → the training data format
4. Verifier → an objective scoreboard
5. Flywheel → agent trains on its own verified successesThe verifier from Part 4 does double duty: it measures whether tuning helped, and it filters the data in Part 5. That’s why it isn’t optional.
Course Structure
| Part | Title | Chapters | What You Build |
|---|---|---|---|
| Part 1 | LoRA With Your Eyes Open | 01–03 | An image LoRA and a contact sheet of every hyperparameter failure mode |
| Part 2 | The Pipeline, Proven | 04–06 | A data → train → fuse → serve loop on Gemma 3 1B |
| Part 3 | Trajectories | 07–09 | A single-tool agent, a trajectory logger, and your first real fine-tune |
| Part 4 | The Verifier | 10–12 | Six sandboxed tools, a held-out eval set, and a baseline table |
| Part 5 | The Flywheel | 13–16 | Rejection sampling → DPO → GRPO → scale up and ship |
Time Budget
| Part | Calendar | Where the time actually goes |
|---|---|---|
| 1 | 1 day | Waiting on training. Read the mflux source meanwhile. |
| 2 | 1 day | Building the Makefile. Worth it. |
| 3 | 2–3 days | Harness design + teacher trace generation |
| 4 | 3–5 days | Writing eval tasks. Nothing else. This is the bottleneck and it’s unavoidable. |
| 5 | Ongoing | Compute, mostly unattended |
Parts 1–2 are a weekend. Part 4 is where the real work is. Part 5 is where the interesting part starts.
The Three Things That Break Projects Like This
1. Skipping the verifier. Without it you’re tuning on vibes and cannot distinguish improvement from noise. It’s also the filter in Part 5 — no verifier, no flywheel. Build it before you need it.
2. Loss masking. Train on tool outputs and the model learns to hallucinate observations rather than fetch them. It will look fine on loss and be useless in the loop. In Part 2 you induce the bug with a canary (train without --mask-prompt, probe for a secret leak), then fix it (--mask-prompt) — never trust a single generate. Verify again in Part 3.
3. Trying to fine-tune knowledge into weights. Codebase facts belong in context. Behavior belongs in weights. Confusing these produces a model that confidently describes an API you deleted last month. The fine-tune is teaching the loop — research, tool use, reporting — and the loop is what small models are actually missing.
How Tutoring Works Here
Use Copy to LLM on any lesson. Assistants should teach in gates: explain the mechanism, quiz you, install and verify the stack runs, prepare data step by step (raw files, then transforms), then train or build. If a tutor dumps every command at once, point them at /llms.txt.
Let’s Go
Part 1: LoRA With Your Eyes Open →
01 lora eyes open
What fine-tuning is for: small, durable behavior changes — and why that is different from prompting.
Understand Dreambooth and training checkpoints, install mflux and smoke-test generation, prepare a raw→512 dataset with captions, then train (and resume) from a JSON config — including what drives wall-clock training time.
Understand what each knob does, check that intuition, then sweep rank, steps, dataset size, targets, and scale — building a contact sheet gate by gate.
02 the pipeline
Understand the toy task and chat JSONL, install mlx-lm and smoke-test it, then generate a train/valid dataset from a script — not by hand.
Understand the three checks (especially loss masking), train with mlx_lm.lora, overfit-check, then deliberately break and fix masking with a canary probe before adapter-vs-merged.
Understand why an OpenAI-compatible server and a Makefile matter, smoke-test the server, then wrap data→train→serve so you can swap datasets without editing the pipeline.
03 trajectories
Understand why you train agentic loop behavior, then build a ~200-line single-tool harness pointed at your OpenAI-compatible server.
Understand trajectory schema and loss masking on tool turns, then wrap the harness so every run emits one JSONL line and a converter to training data.
Understand distillation metrics, generate teacher trajectories, convert and train with the existing Makefile, then measure tool-call validity before vs after.
04 the verifier
Understand why verifiers and small toolsets matter, then expand the harness to six sandboxed tools and smoke-test each one.
Understand held-out Type A vs Type B tasks, then build eval/tasks/ interactively with a Type B majority and mechanical verification.
Understand metrics and the uncomfortable baseline question, then build eval.py, smoke-test it, and fill a multi-model baseline table.
05 the flywheel
Understand rejection sampling / STaR, check the idea, then wire generate→verify→filter→train→score one gate at a time using existing harness pieces.
Understand DPO and GRPO as ways to use failures, check the ideas, then add preference pairs only after a rejection-sampling baseline exists.
Understand Reflexion notes and skill accretion as zero-GPU learning, check the idea, then wire a notes file and one skill into the harness.
Understand scale-up options, check the plan, then move base model size, optional Thor/Unsloth rebuild, and Ollama serve — same harness, one URL change.