Understand why you train agentic loop behavior, then build a ~200-line single-tool harness pointed at your OpenAI-compatible server.

Chapter 07: The Harness

Objective

Build the smallest possible agent harness (~200 lines, one tool) pointed at the Chapter 06 server. Goal of Part 3 overall: fine-tune the 1B until it emits syntactically valid tool calls — not smart ones, valid ones.

Concept: Behavior, not knowledge

You are not fine-tuning the model on your code. You are fine-tuning it on the shape of agentic behavior.

Codebase knowledge belongs in context at runtime — bake it into weights and you retrain when the code changes, and it hallucinates moved APIs. What small models are bad at, and what is stably trainable, is the loop:

take a task → emit a well-formed tool call → read the result → reason → call again → stop when done → report

That procedure is learnable. That is what we train.

The minimal harness

One tool:

read_file(path) -> string

Loop:

System prompt describing the tool and output protocol
User task
Model emits a tool call
Harness parses, executes, appends result
Repeat until final answer or step cap

Point it at the Chapter 06 OpenAI-compatible endpoint.

Protocol rule

Use a simple protocol (JSON block or XML tags) you can parse with a regex and validate. The system prompt must be identical between training and inference. Mismatch silently destroys everything downstream.

Concept check

Q1. Why train the loop (tool-call shape, stop when done) instead of embedding the whole repo into the weights?

Q2. Why must the system prompt match exactly between training data and live harness runs?

Concept answer key — attempt first

Answer key (concept)

Q1

Model answer: Repo facts go stale and belong in context; the stable trainable skill is agentic procedure (call tools, use results, stop).

Pass criteria: behavior vs knowledge; staleness or retrieval

Q2

Model answer: The model learns the protocol from the system text; a different prompt at inference is a different task and validity collapses.

Pass criteria: train/serve protocol identity

Gate: Confirm the server is up

Smoke-test the Chapter 06 endpoint with a simple curl chat completion. Pass: HTTP 200 and a completion body.

If the server is down, fix that before writing harness code.

Gate: Build the one-tool harness

Implement harness.py (name flexible):

OpenAI-compatible client → your local URL
Parse tool calls for read_file only
Execute against a small sandbox directory (not your whole home folder)
Step cap
Print the final answer

Pass: one hand-written task like “What is the first line of README.md?” completes with a real read_file call when you force or mock a good model response — or when using a strong teacher URL if you already have one.

Walk through parse failures with your tutor; do not jump to logging yet.

Check your understanding

Q3. Why start with one tool instead of the full coding toolset?

Q4. “Valid tool call” vs “correct answer to the user’s question” — which is Part 3’s primary metric, and why start there?

Answer key — attempt every question first

Answer key

Q3

Model answer: Each tool multiplies protocol surface and required examples; one tool proves the loop and logging path first.

Pass criteria: protocol surface / complexity control

Q4

Model answer: Validity first — syntax/protocol is the wide-open gap on small models; correctness comes after the model can speak tool language.

Pass criteria: validity/protocol before smart answers

← Part 3 Index · Next: Chapter 08 →