Chapter 08: The Logger

Wrap the harness so every run emits one JSONL trajectory. The schema decisions here — outcome field, masking, logging failures — are load-bearing for Parts 4 and 5.

TUTOR WITH THEFOCUS.AI

Agent Integration

Copy this prompt into Claude, ChatGPT, or any external AI assistant. It points the assistant to the course instructions and links it to your student profile to track your progress and customize observations.

Please tutor me in this lesson using the following context. First, read the instructions at: https://courses.thefocus.ai/llms.txt My Student ID is: <none> The lesson markdown source is at: https://courses.thefocus.ai/fine-tune-local-agent/03-trajectories/08-the-logger.md

You are not enrolled yet. Enroll to generate a Student ID to track lesson completions and store learning notes.

Chapter 08: The Logger

The logger is the actual artifact of Part 3. Wrap the harness so every run emits one JSONL line:

{
  "task_id": "...",
  "task": "What does the retry logic in client.py do?",
  "messages": [
    {"role": "system", "content": "..."},
    {"role": "user", "content": "..."},
    {"role": "assistant", "content": "<thinking>...</thinking><tool>read_file('client.py')</tool>"},
    {"role": "tool", "content": "<file contents>"},
    {"role": "assistant", "content": "The retry logic uses exponential backoff..."}
  ],
  "outcome": null,
  "steps": 3,
  "meta": {"model": "...", "timestamp": "..."}
}

Design Notes That Will Bite You Later If You Skip Them

outcome is null for now. Part 4’s verifier fills it. Leave the field.
Every tool role message is masked out of the loss. This is Check 2 from Chapter 05, now load-bearing. The model must be trained to produce the assistant turns and to condition on the tool turns — never to produce them.
Log failures too. Part 5’s preference-learning variant needs them. Storage is free.
Version the schema. You will change it.

The Converter

Write to_training_data.py: trajectory JSONL → mlx-lm chat JSONL, with masking applied. This is the bridge between “what the agent did” and “what the model trains on,” and it’s the file you’ll edit most in Part 5.

Because you built the generator-script habit in Chapter 04, regenerating the entire training set from raw trajectories should be one command.

← Chapter 07 · Next: Chapter 09 →