Understand why an OpenAI-compatible server and a Makefile matter, smoke-test the server, then wrap data→train→serve so you can swap datasets without editing the pipeline.

Chapter 06: Serve and Automate

Objective

Serve the tuned model over an OpenAI-compatible HTTP API and wrap data → train → fuse → serve → eval in a Makefile (or justfile) you will reuse for the rest of the course.

Concept: One protocol, many models

Part 3’s harness will talk to whatever sits at a chat-completions URL. If you standardize on:

POST /v1/chat/completions

then local mlx-lm, later Ollama, or a remote teacher can share the same client. Scale-up in Part 5 becomes one URL change, not a rewrite.

The Makefile is the pipeline. Part 3 changes only JSONL content and re-runs targets. Part 5 runs the loop unattended. If swapping data requires editing five scripts, the pipeline is not done.

Concept check

Q1. Why serve behind an OpenAI-compatible API instead of only calling mlx_lm.generate from Python for the rest of the course?

Q2. What does “swap the dataset twice without touching any other file” prove about your Makefile?

Concept answer key — attempt first

Answer key (concept)

Q1

Model answer: Same client/harness for local and remote models; Part 3+ can point at different backends with one URL; matches production agent shape.

Pass criteria: interchangeable backends / one harness / URL swap

Q2

Model answer: Data is decoupled from train/serve wiring — the pipeline is parameterized by data path or generator only.

Pass criteria: decoupling / data as the only moving part

Gate: Serve and smoke-test HTTP

mlx_lm.server --model ./models/gemma-json-1b --port 8080
# or with adapter path if you did not fuse

In another terminal:

curl http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "default",
    "messages": [{"role": "user", "content": "What is the capital of France?"}],
    "max_tokens": 100
  }'

Pass: JSON response; with a successful Chapter 05 tune, assistant content should look like your JSON habit.

Fix server errors before automating.

Gate: Makefile targets

Create a Makefile or justfile with at least:

data:     # regenerate JSONL from generator script
train:    # mlx_lm.lora ... --mask-prompt  (always for chat JSONL)
overfit:  # 10-example sanity check
canary:   # optional: data-canary + unmasked vs masked A/B (see Ch 05)
fuse:     # merge adapters
serve:    # mlx_lm.server
eval:     # held-out prompts, print results

Wire real commands underneath. make train must pass --mask-prompt so production trains do not reintroduce the Check 2 failure mode. Run:

make data && make train && make serve

from clean (or document what “clean” means). Target: under ~15 minutes on your box for the toy task.

Gate: Swap data without rewriting the pipeline

Change seed questions / generator inputs only.
make data && make train
Confirm serve/eval still work.

Exit criteria

make data && make train && make serve works from clean in under fifteen minutes
Served model reliably returns valid JSON for the toy task
Dataset swapped at least twice without editing train/serve wiring

Check your understanding

Q3. Part 5 will run this loop hundreds of times. Which artifact is more important long-term: one strong adapter file, or a reliable Makefile?

Q4. The harness in Part 3 should hardcode mlx_lm.generate calls — true or false? Why?

Answer key — attempt every question first

Answer key

Q3

Model answer: The Makefile (pipeline). Adapters are disposable; the loop must be re-runnable.

Pass criteria: pipeline/Makefile over a single artifact

Q4

Model answer: False — call the OpenAI-compatible HTTP API so backends can change.

Pass criteria: false + API/URL abstraction

← Chapter 05 · Next: Part 3 →