---
title: "Chapter 10: Real Tools"
description: "Understand why verifiers and small toolsets matter, then expand the harness to six sandboxed tools and smoke-test each one."
type: lesson
order: 10
chapter: "04-the-verifier"
---

# Chapter 10: Real Tools

## Objective

Expand to a **six-tool** sandboxed set. Later chapters add the eval set and scoreboard; here you grow the protocol surface carefully and prove each tool works.

---

## Concept: No verifier, no flywheel

Without a mechanical pass/fail you cannot answer “did the fine-tune help?” — you tune on vibes. Part 5 **filters** trajectories by success. No verifier → no filter → no flywheel.

Coding is ideal: **tests pass or they don’t.** Exit code. No reward model required.

### Toolset (stop at six)

```text
read_file(path)
list_dir(path)
grep(pattern, path)
edit_file(path, old, new)
run_tests(target)
git_diff()
```

**Resist adding more.** Each tool multiplies protocol surface and the examples the small model needs.

### Sandbox

Sandbox write/execute (`edit_file`, `run_tests`): container, git worktree, or disposable copy. The agent *will* do something stupid; you want `git checkout .` and move on.

---

## Concept check

**Q1.** Why is “tests pass / exit code” a better core filter for Part 5 than an LLM-as-judge score alone?

**Q2.** Why not add fifteen tools “for realism” on day one?

<details>
<summary>Concept answer key — attempt first</summary>

## Answer key (concept)

### Q1

**Model answer:** Mechanical, uncheatable, free filter for the flywheel; LLM judges are soft and gameable.

**Pass criteria:** mechanical / binary / flywheel filter

### Q2

**Model answer:** Each tool multiplies protocol + training data needs; small models drown in surface area.

**Pass criteria:** protocol surface / data cost

</details>

---

## Gate: Implement tools behind a sandbox root

Extend the harness. All paths resolve under a sandbox directory. `run_tests` and `edit_file` cannot escape it.

**Pass:** unit-level smoke for each tool (call from a tiny script or REPL): list, read, grep, edit, tests, diff.

---

## Gate: One multi-tool task by hand

Run a single task that needs at least three tools (e.g. find a function, edit, run a test). Log a trajectory.

**Pass:** trajectory shows multiple tool types; sandbox survived (repo not destroyed).

---

## Check your understanding

**Q3.** Which two tools are most important to sandbox, and why?

**Q4.** How does a larger toolset change what the **logger/converter** must handle?

<details>
<summary>Answer key — attempt every question first</summary>

## Answer key

### Q3

**Model answer:** `edit_file` and `run_tests` — they mutate state and execute code.

**Pass criteria:** write + execute named

### Q4

**Model answer:** More tool-call shapes in messages; masking still applies to all tool results; schema may need tool names/args validated.

**Pass criteria:** more protocol variants + still mask tool outputs

</details>

---

[← Part 4 Index](/fine-tune-local-agent/04-the-verifier/) · [Next: Chapter 11 →](/fine-tune-local-agent/04-the-verifier/11-the-eval-set/)