Chapter 07: One Agent, One Reviewed Change
Pin an agent CLI, inspect its permissions, and run one supervised implementation attempt.
TUTOR WITH THEFOCUS.AI
Copy this prompt into Claude, ChatGPT, or any external AI assistant. It points the assistant to the course instructions and links it to your student profile to track your progress and customize observations.
You are not enrolled yet. Enroll to generate a Student ID to track lesson completions and store learning notes.
Chapter 07: One Agent, One Reviewed Change
Objective
Pin an agent CLI, inspect its permissions, and run one supervised implementation attempt.
Your tutor should stop at each gate and wait for your evidence before continuing.
Concept
The agent executable belongs in your reproducibility record, alongside Nix inputs and test commands. Its model is a separate service: pinning the CLI cannot freeze server-side weights, routing, sampling, availability, or account policies. Record the exact model ID and run date rather than promising deterministic model output.
We use Claude Code as one concrete adapter. The workflow also applies to other agents, but their flags and authorization models differ. Do not translate flags by guesswork. Numtide’s llm-agents.nix packages several CLIs; each package has its own licensing and platform constraints. Its current documented platform set includes Apple Silicon and ARM/x86 Linux. Intel Mac users should verify the particular package before using this route.
First ask the agent to inspect the project with read-only tools. Then run the edit interactively so you can see permission requests. Unattended shell access belongs in a disposable VM, introduced in Chapter 09.
Concept check
Q1. What remains variable after pinning a CLI source revision?
Q2. Why do we start with a read-only prompt?
Concept answer key — attempt first
Answer key (concept)
Q1. Model service behavior, credentials, runtime state, and host details. Pass criteria: pinning the CLI does not pin the model.
Q2. It verifies authentication and context without granting editing or shell execution. Pass criteria: separate connection from mutation.
Gate: Resolve an agent package once
From the project’s devenv shell:
git ls-remote https://github.com/numtide/llm-agents.nix.git refs/heads/mainReview the returned revision and its package definition. Put only the full commit hash in agent-revision.txt. This is a deliberate bootstrap lookup, not something to repeat on every run. Build that revision:
agent_rev=$(cat agent-revision.txt)
nix build "github:numtide/llm-agents.nix/$agent_rev#claude-code" --out-link .agent-cli
export PATH="$PWD/.agent-cli/bin:$PATH"
claude --version
claude --helpAdd .agent-cli to .gitignore. Commit agent-revision.txt; rebuild from it on every other machine. Review any proposed cache key before trusting it. A cache is a source of executable software. The package is unfree; use it only under terms you are entitled to accept. If the package is unavailable for your architecture, use a supported Linux VM rather than silently substituting an unrecorded host installation.
Gate: Verify authentication and context
Use the CLI’s normal interactive login for your account, or inject a scoped API key through your secret manager. Do not put a key in Nix expressions, prompts, lockfiles, shell history, or a Git-tracked file.
Choose an available exact model ID and record it in your run notes. For the scripted read-only smoke test, current Claude documentation recommends --bare, which skips automatic project and user configuration and uses API-key authentication rather than a subscription login:
export LAB_MODEL='replace-with-your-available-exact-model-id'
claude --bare -p --model "$LAB_MODEL" --max-turns 3 --max-budget-usd 0.25 --allowedTools Read 'Read AGENTS.md and specs/completion.md. Describe the test command and allowed changes. Do not edit.'The placeholder is intentional: model access and prices vary by account. Verify --bare, budget, and turn flags in your installed version’s help before proceeding. Set a provider-side spending limit too; the CLI’s cost estimate is not a hard billing guarantee.
Pass: the agent identifies the actual spec and final test command, and the working tree is unchanged. If you use subscription authentication, do this small read-only step interactively instead of removing --bare from a script without reviewing what would auto-load.
Gate: Make one supervised change
Commit the reviewed agent revision and ignore rule before starting. Run claude --model "$LAB_MODEL" interactively and paste the contents of PROMPT.md. Approve only the operations needed for this exercise; stop if it asks to change tests or install host packages.
When it exits:
git status --short
git diff --stat
git diff -- taskboard.py IMPLEMENTATION_PLAN.md
devenv testPass: only the allowed files changed, tests pass unchanged, and you can explain how the implementation handles duplicates and aliases. The owner commits after review. Save this successful branch; use a separate clone of the exercise baseline for the looping experiment.
Known failure modes
- A version pin may be too old to support current flags; deliberately update and review it.
AGENTS.mdmight not be auto-read by a given CLI; the prompt explicitly requests it.- A successful process exit says the agent finished its run, not that the task meets acceptance.
- Tool allowlists and prompts are not OS isolation. Bash can invoke arbitrary programs.
Check your understanding
Q3. What do you review besides the agent’s summary?
Q4. Why keep the first successful change before experimenting with loops?
Answer key — attempt every question first
Answer key
Q3. The complete diff, protected files, real test output, and behavior. Pass criteria: independent evidence.
Q4. It provides a known-good comparison without overwriting work. Pass criteria: use a separate branch/clone baseline.