Chapter 16: Scale Up and Ship
Understand scale-up options, check the plan, then move base model size, optional Thor/Unsloth rebuild, and Ollama serve — same harness, one URL change.
TUTOR WITH THEFOCUS.AI
Copy this prompt into Claude, ChatGPT, or any external AI assistant. It points the assistant to the course instructions and links it to your student profile to track your progress and customize observations.
You are not enrolled yet. Enroll to generate a Student ID to track lesson completions and store learning notes.
Chapter 16: Scale Up and Ship
Objective
Once the flywheel turns on 1B: scale the base model, optionally rebuild on CUDA (Jetson Thor / Unsloth), ship via Ollama, and automate flywheel.py with an eval-score chart.
Concept: Same harness, bigger engine
- Move up the ladder (4B → 12B…): same Makefile, more layers/longer runs. Baseline table (Ch 12) tells you if it paid off.
- Thor / CUDA: Unsloth rebuild. Nemotron notes: BF16 (not casual 4-bit QLoRA), pin stack packages, do not fine-tune router on MoE variants.
- Ship: merge → GGUF → Ollama; point harness at new URL — payoff for OpenAI-compatible API in Ch 06.
Final deliverable
flywheel.py: generate → verify → filter → train → score; append results row; stop when improvement < threshold. Chart: eval score vs iteration.
Exit: round n+1 > n on held-out for ≥2 consecutive rounds, no teacher — local, fine-tuned, improving on verified experience.
Concept check
Q1. Why can scaling the base model reuse the Chapter 06 Makefile with minimal edits?
Q2. What is the payoff of standardizing on /v1/chat/completions when you move from mlx-lm to Ollama?
Concept answer key — attempt first
Answer key (concept)
Q1
Model answer: Pipeline is model-name/path parameterized; data→train→serve loop unchanged.
Pass criteria: parameterized pipeline / same stages
Q2
Model answer: Harness client unchanged — only the base URL (and maybe model name) changes.
Pass criteria: one URL / same client
Gate: Decide scale target with the scoreboard
Using your Ch 12 table, pick the next base size (or stay) with a one-sentence rationale to your tutor.
Gate: One larger-model train + eval
Run the existing pipeline on the larger model (Mac MLX if it fits; else plan Thor). Compare held-out scores to 1B.
Gate: Ship path (when ready)
Merge → GGUF → ollama create / serve → harness URL swap → smoke-test one task.
Pass: harness completes a task against the shipped endpoint.
Gate: flywheel.py + chart
Automate multi-round loop; produce score-vs-iteration chart.
Pass: two automated rounds or a dry-run that writes the results table format.
Check your understanding
Q3. Your 1B flywheel works but 12B prompted base wins the table. What is a rational next step?
Q4. Name the two course constraints that remain true after ship.
Answer key — attempt every question first
Answer key
Q3
Model answer: Fine-tune the larger model (or accept prompting) — do not only polish 1B; use the table.
Pass criteria: move tune up or accept scale result
Q4
Model answer: Local weights; model is fine-tuned (not only prompted).
Pass criteria: local + fine-tuned