Understand trajectory schema and loss masking on tool turns, then wrap the harness so every run emits one JSONL line and a converter to training data.

Chapter 08: The Logger

Objective

Wrap the harness so every run emits one JSONL trajectory. Schema choices here — especially outcome and masking tool turns — are load-bearing for Parts 4 and 5.

Concept: Trajectories are the training ore

The logger is the real artifact of Part 3. Each run becomes one line, roughly:

{
  "task_id": "...",
  "task": "What does the retry logic in client.py do?",
  "messages": [
    {"role": "system", "content": "..."},
    {"role": "user", "content": "..."},
    {"role": "assistant", "content": "<thinking>...</thinking><tool>read_file('client.py')</tool>"},
    {"role": "tool", "content": "<file contents>"},
    {"role": "assistant", "content": "The retry logic uses exponential backoff..."}
  ],
  "outcome": null,
  "steps": 3,
  "meta": {"model": "...", "timestamp": "..."}
}

Design notes that bite you later

outcome is null for now. Part 4’s verifier fills it. Leave the field.
Every tool role message is masked out of the loss. Chapter 05 Check 2, now load-bearing: train the model to produce assistant turns and condition on tool turns — never to produce tool contents.
Log failures too. Part 5 preference learning needs them.
Version the schema. You will change it.

The converter

to_training_data.py: trajectory JSONL → mlx-lm chat JSONL with masking. Bridge between “what the agent did” and “what trains.” One command regenerates the set (same habit as Chapter 04’s generator).

Concept check

Q1. Why leave outcome: null in the schema before you have a verifier?

Q2. If tool-role contents are included in the training loss, what bad behavior do you teach?

Concept answer key — attempt first

Answer key (concept)

Q1

Model answer: So Part 4 can fill the field without a schema break; the flywheel and scoreboard need a stable place for pass/fail.

Pass criteria: forward compatibility / Part 4 fills it

Q2

Model answer: Hallucinating file contents / tool observations instead of calling tools.

Pass criteria: hallucinate observations

Gate: Emit one trajectory

Instrument the harness: each completed (or failed) run appends one JSON object line to trajectories/raw.jsonl (name flexible).

Pass: run two tasks; file has two lines; each has messages including at least one tool turn when tools were used.

Gate: Converter smoke test

Implement to_training_data.py → writes mlx-lm chat JSONL with tool turns excluded from loss (per your toolchain’s masking approach).

Pass: make data (or one command) regenerates train JSONL from trajectories; spot-check that assistant targets look right and tool blobs are not trained as generations.

Check your understanding

Q3. Why log failed trajectories, not only successes?

Q4. Raw trajectories vs training JSONL — which is the source of truth you keep regenerating from?

Answer key — attempt every question first

Answer key

Q3

Model answer: Preference methods (DPO) need rejected/failed pairs; failures are signal, not trash.

Pass criteria: preference / DPO / learning from failures

Q4

Model answer: Raw trajectories; training JSONL is a derived artifact from the converter.

Pass criteria: trajectories as source; converter regenerates

← Chapter 07 · Next: Chapter 09 →