Fine-Tune a Self-Improving Local Coding Agent

A five-module ladder, Apple Silicon primary, Jetson Thor for scale.

5 parts · 16 chapters · Will Schenk

The Constraints (fixed)

1. The model is fine-tuned. Not just prompted, not just retrieved-into.
2. The model is local. Weights on disk, inference on your hardware.

Everything in this course serves those two. Retrieval, tool design, and context management appear only where they generate training data or gate a deliverable.

What You’ll Build

By the end of this course you’ll have:

• Calibrated intuition for LoRA hyperparameters — rank, alpha, steps, dataset size — built where you can see the failure modes
• A repeatable data → train → fuse → serve pipeline that runs in minutes
• An agent harness that logs every trajectory in a trainable format
• A fine-tuned small model that emits valid tool calls where the base model can’t
• A verifier: a binary, mechanical, uncheatable pass/fail scoreboard on real coding tasks
• A flywheel: an agent that attempts tasks, keeps only verified successes, retrains on them, and scores higher each round — with no teacher in the loop

Hardware Split

Model choice on the Mac: Gemma 3. Nemotron’s hybrid Mamba2-Transformer architecture is still maturing in mlx-lm — it runs for inference but fights you for training. Save it for the Thor. Gemma 3’s ladder (270M → 1B → 4B → 12B → 27B) is exactly the ramp this course wants.

The Spine

Each module produces an artifact the next one consumes. Nothing is decorative.

1. mflux LoRA        → intuition about the knobs
2. Gemma 1B toy tune → a working train→fuse→serve pipeline
3. Trajectory logger → the training data format
4. Verifier          → an objective scoreboard
5. Flywheel          → agent trains on its own verified successes

The verifier from Part 4 does double duty: it measures whether tuning helped, and it filters the data in Part 5. That’s why it isn’t optional.

Course Structure

Time Budget

Parts 1–2 are a weekend. Part 4 is where the real work is. Part 5 is where the interesting part starts.

The Three Things That Break Projects Like This

1. Skipping the verifier. Without it you’re tuning on vibes and cannot distinguish improvement from noise. It’s also the filter in Part 5 — no verifier, no flywheel. Build it before you need it.

2. Loss masking. Train on tool outputs and the model learns to hallucinate observations rather than fetch them. It will look fine on loss and be useless in the loop. Check it in Part 2, verify it again in Part 3.

3. Trying to fine-tune knowledge into weights. Codebase facts belong in context. Behavior belongs in weights. Confusing these produces a model that confidently describes an API you deleted last month. The fine-tune is teaching the loop — research, tool use, reporting — and the loop is what small models are actually missing.

Let’s Go

Part 1: LoRA With Your Eyes Open →

hey@thefocus.ai · thefocus.ai