Chapter 06: Serve and Automate

Serve the tuned model over the OpenAI-compatible API and wrap the entire loop in a Makefile you'll use for the rest of the course.

TUTOR WITH THEFOCUS.AI

Agent Integration

Copy this prompt into Claude, ChatGPT, or any external AI assistant. It points the assistant to the course instructions and links it to your student profile to track your progress and customize observations.

Please tutor me in this lesson using the following context. First, read the instructions at: https://courses.thefocus.ai/llms.txt My Student ID is: <none> The lesson markdown source is at: https://courses.thefocus.ai/fine-tune-local-agent/02-the-pipeline/06-serve-and-automate.md

You are not enrolled yet. Enroll to generate a Student ID to track lesson completions and store learning notes.

Chapter 06: Serve and Automate

Serving

mlx_lm.server --model ./models/gemma-json-1b --port 8080

Now anything that speaks the OpenAI chat-completions API can hit http://localhost:8080/v1/chat/completions. This endpoint is what Part 3’s harness talks to. Same protocol, one URL — which is also why the final scale-up in Part 5 is a one-line change.

Deliverable: The Makefile

A Makefile or justfile:

make data     # regenerate JSONL from source
make train    # mlx_lm.lora
make overfit  # the 10-example check
make fuse     # merge
make serve    # mlx_lm.server
make eval     # generate on held-out prompts, print results

This is not busywork. Part 3 changes only the JSONL content and reruns everything. Part 5 runs this loop hundreds of times unattended. The Makefile is the pipeline.

Exit Criteria

make data && make train && make serve from clean, in under fifteen minutes
The served model always returns valid JSON
You have swapped the dataset at least twice without touching any other file

That last one is the real test. If swapping data requires edits elsewhere, the pipeline isn’t done.

← Chapter 05 · Next: Part 3 →