Part 5: The Flywheel — Rejection Sampling to Self-Improvement

Chapters 13–16: Close the loop. The agent attempts tasks, the verifier filters to verified successes, the model retrains on them — and improves with no teacher.

TUTOR WITH THEFOCUS.AI

Agent Integration

Copy this prompt into Claude, ChatGPT, or any external AI assistant. It points the assistant to the course instructions and links it to your student profile to track your progress and customize observations.

Please tutor me in this lesson using the following context. First, read the instructions at: https://courses.thefocus.ai/llms.txt My Student ID is: <none> The lesson markdown source is at: https://courses.thefocus.ai/fine-tune-local-agent/05-the-flywheel/13-rejection-sampling.md

You are not enrolled yet. Enroll to generate a Student ID to track lesson completions and store learning notes.

Part 5: The Flywheel

Rejection sampling → self-improvement · ongoing

Close the loop. The agent attempts tasks, the verifier filters to verified successes, those successes become training data, the model is retrained, redeployed, and re-scored. Repeat. The agent now learns from its own experience, not a teacher’s. Both constraints satisfied: local, fine-tuned.

        ┌─────────────────────────────────────────┐
        │                                         │
        ▼                                         │
  [ Generate ]  agent attempts N tasks            │
        │       (temperature > 0, k samples each) │
        ▼                                         │
  [ Verify ]    run_tests → pass/fail             │
        │                                         │
        ▼                                         │
  [ Filter ]    keep only verified successes      │
        │                                         │
        ▼                                         │
  [ Train ]     LoRA on winning trajectories ─────┘
        │
        ▼
  [ Score ]     eval.py on held-out set

Chapter	What You’ll Do
13 · Rejection Sampling	Spin the loop: sample, verify, filter, train, score
14 · Learning From Failures	DPO on chosen/rejected pairs, then GRPO with your verifier as the reward function
15 · The No-Weights Complement	Reflexion notes and skill accretion — learning that costs zero GPU
16 · Scale Up and Ship	Move up the Gemma ladder, rebuild on the Thor, serve via Ollama

What You’ll Have After Part 5

flywheel.py — one command runs generate → verify → filter → train → score
A results table that grows a row per iteration
A chart: eval score vs. iteration. That chart is the entire project, in one image.

Chapter 13 →