Chapter 10: Real Tools
Expand from one tool to six — sandboxed — and understand why every additional tool multiplies the protocol surface your small model has to learn.
TUTOR WITH THEFOCUS.AI
Copy this prompt into Claude, ChatGPT, or any external AI assistant. It points the assistant to the course instructions and links it to your student profile to track your progress and customize observations.
You are not enrolled yet. Enroll to generate a Student ID to track lesson completions and store learning notes.
Chapter 10: Real Tools
Objective
Expand to a real toolset, then build the thing everything else depends on: an evaluation harness that produces a binary, mechanical, uncheatable pass/fail on real tasks.
Why This Is the Module People Skip
Without a verifier you cannot answer “did the fine-tune help?” — so you tune on vibes, and vibes on a 1B model are indistinguishable from noise. Worse: Part 5’s entire premise is filtering trajectories by whether they succeeded. No verifier, no filter. No filter, no flywheel.
Coding is the ideal domain for this precisely because success is mechanically checkable. Tests pass, or they don’t. You do not need a reward model, an LLM judge, or human labels. You need a test suite and an exit code. Every other domain would kill for this.
The Toolset
read_file(path)
list_dir(path)
grep(pattern, path)
edit_file(path, old, new)
run_tests(target)
git_diff()Six tools. Resist adding more. Each one multiplies the protocol surface the small model has to learn, and every tool you add needs training examples covering it.
Sandbox the Dangerous Ones
Sandbox the write and execute tools (edit_file, run_tests). Container, git worktree, whatever — the agent will do something stupid and you want to git checkout . and move on.