Chapter 16: Scale Up and Ship
Move the base model up the Gemma ladder, rebuild the tune on the Jetson Thor under Unsloth/CUDA, and serve via Ollama. Same harness, one URL changed.
TUTOR WITH THEFOCUS.AI
Copy this prompt into Claude, ChatGPT, or any external AI assistant. It points the assistant to the course instructions and links it to your student profile to track your progress and customize observations.
You are not enrolled yet. Enroll to generate a Student ID to track lesson completions and store learning notes.
Chapter 16: Scale Up and Ship
Once the flywheel demonstrably turns on 1B:
1. Move the Base Model Up
4B → 12B. Same pipeline, bigger --num-layers, longer runs. The Makefile from Chapter 06 doesn’t change; the baseline table from Chapter 12 tells you whether the move paid for itself.
2. Move to the Thor
Rebuild the tune under Unsloth/CUDA. If you go Nemotron, three things to know going in:
- Hybrid architecture means BF16 — 4-bit QLoRA isn’t directly supported
- Pin
mamba_ssmandcausal_conv1don the CUDA 12.8 stack - Do not fine-tune the router layer on the MoE variants — Unsloth disables it by default for stability, and that default is correct
3. Ship
Merge → GGUF → Ollama. Serve on the Thor. Point the harness at it. Same harness, one URL changed — the payoff for standardizing on the OpenAI chat-completions API back in Chapter 06.
The Final Deliverable
flywheel.py — one command runs generate → verify → filter → train → score, appends a row to a results table, and stops when improvement falls below a threshold.
And a chart: eval score vs. iteration. That chart is the entire project, in one image.
Exit Criteria
Round n+1 scores higher than round n on held-out tasks, for at least two consecutive rounds, with no teacher in the loop. A local, fine-tuned model — both constraints satisfied — improving on its own verified experience.