---
title: "Self Improving"
---

# 07 — The Self-Improving Loop

## The Big Idea

The prompt runner is useful, but it's still manual. You write a prompt, you run it, you look at the output, you tweak the prompt. What if we could **close the loop** — feed the errors back automatically?

This is the core insight of agentic AI: **the model can improve its own output when given feedback.**

## How It Works

The self-improving loop has a "rerun" mode. Instead of just running `prompt.txt` fresh, it:

1. Takes the **original prompt**
2. Takes the **failed test output**
3. Takes the **generated code**
4. Combines them with a human-readable error message
5. Feeds everything back to Ollama
6. Gets improved code

The key is that a human provides the error message, but the model sees the full context: what we asked for, what it generated, and what went wrong.

## The Script

This is `run_prompt.sh` with rerun support added (generated by our `3-prompt` exercise):

```bash
#!/bin/bash

DIR=$1
MODE="regular"
MESSAGE=""

if [ -z "$DIR" ]; then
    echo "Usage: $0 <directory> [--rerun <message>]"
    exit 1
fi

if [ "$2" == "--rerun" ]; then
    MODE="rerun"
    MESSAGE="$3"
fi

if [ "$MODE" == "rerun" ]; then
    ./1-combind/code \
        -m "$MESSAGE" \
        "$DIR/prompt.md" "$DIR/test-output.md" "$DIR/code" | \
        ollama run gemma4:26b --nowordwrap | \
        tee "$DIR/ollama-output.md"
else
    cat "$DIR/prompt.md" | ollama run gemma4:26b --nowordwrap | tee "$DIR/ollama-output.md"
fi

./2-extract/code "$DIR"

if [ -f "$DIR/test" ]; then
    chmod +x "$DIR/test"
    "$DIR/test" 2>&1 | tee test-output.md
fi
```

## The Loop in Action

Here's a real session from the author's walkthrough:

### Run 1: It Fails

```bash
./run_prompt.sh 2-runner
```

Everything looks good, but when the test runs:

```
regression_test.sh: line 48: the: command not found
```

The generated code has weird artifacts — garbage characters at the end of some lines. It's not clean.

### Feedback

The author copies the error and the generated files back into the chat:

```
when i ran this with the runner itself, I got these results and it didn't work.
python3 runs/run_002/generated_code_001.py
  File "...generated_code_001.py", line 42
    with open(prompt_path, "w", encoding="cap=utf-8") as f:
                                               ^
SyntaxError: unterminated string literal (detected at line 42)
```

### The Model Fixes It

The model recognizes specific failure modes:

- `encoding="cap=utf-8"` — it invented a bad argument (should be `encoding="utf-8"`)
- Garbage characters — likely from not using `re.DOTALL` in regex

It produces an updated specification (plan2.md) with explicit constraints:

- _"No Syntax Hallucinations: Ensure all Python syntax is standard"_
- _"Ensure open() arguments are correctly formatted"_
- _"No Code Fragmentation"_

### Run 2: Better, But Still Not Perfect

The improved prompt produces better code, but there are still issues. More feedback, more improvement. This is the evolution we traced in Spell #3: plan1 → plan2 → plan3 → plan4.

## The Master Prompt (plan4.md)

After several iterations, here's the final, bulletproof prompt that consistently produces working code. This is what you'd use as your "one-shot" for generating the Prompt Iterancy Tracker:

---

# Master Prompt: Prompt Iterancy Tracker (PIINT)

**Role:** You are an expert Python Developer specializing in automation, CLI tools, and mission-critical code integrity. Your goal is to write code that is syntactically perfect, follows PEP 8 standards, and handles all edge cases without fragmentation or syntax errors.

**Objective:** Build a "Prompt Iterancy Tracker" (MVP version) that automates the process of running prompts through a local Ollama instance, extracting generated code, and versioning the results in a structured local directory.

**Technical Requirements:**

1. **Robust Input Handling:**
   - **Existence Check:** Check if `prompt.txt` exists in the root directory.
   - **Auto-Creation:** If `prompt.txt` is missing, automatically create it with the default text: `"Write a python script that prints 'Hello World'"`.
   - **Encoding Standard:** Every `open()` function call **must** explicitly include `encoding="utf-8"`.

2. **Core Workflow & API Integration:**
   - **Library:** Use `requests`.
   - **Endpoint:** Target `http://localhost:11434/api/generate` using the model `gemma4:26b`.
   - **Payload:** Set `"stream": False`.
   - **Error Handling:** Wrap the API call in a `try-except` block. If the connection fails, print `"Error: Is Ollama running?"` and exit via `sys.exit(1)`.

3. **Versioning & File Management:**
   - **Directory Structure:** Create a `runs/` directory. Inside, create uniquely numbered subdirectories (e.g., `run_001/`, `run_002/`).
   - **Auto-Increment Logic:** Scan the `runs/` folder, identify the highest existing number in `run_XXX`, and increment it by 1.
   - **Data Preservation:** Inside each `run_XXX/` folder, save:
     1. A copy of `prompt.txt` used.
     2. The `full_response.txt` (raw text).
     3. All extracted code files.

4. **Code Extraction (Regex):**
   - **Regex Engine:** Use `re.findall` with `re.DOTALL`. The pattern must target: `r"```(\w*)\n(.*?)\n```"`.
   - **Language Mapping:** Use a dictionary to map Markdown tags to extensions (e.g., `python` → `.py`). Default to `.txt` if unknown.
   - **Multi-Block Support:** Extract and save every code block found as a separate numbered file (e.g., `generated_code_001.py`).

**DEFENSIVE CODING & ANTI-HALLUCINATION (CRITICAL):**

1. **Zero-Tolerance for Syntax Hallucination:**
   - **Comma Integrity:** Double-check every dictionary (`{}`) and function call for missing commas.
   - **Argument Accuracy:** Use only standard Python arguments. (e.g., `open(file, "w", encoding="utf-8")`. Do **not** invent arguments like `cap=utf-8`).
   - **OS Module Standard:** Use only standard `os.makedirs` arguments (e.g., `exist_ok=True`).
2. **Variable Consistency:** Ensure all constant names (e.g., `MODEL_NAME`) are used with identical casing throughout the entire script. No `Model_Name` or `model_name` variations.
3. **No Complex Syntax:** Avoid "clever" Python features like the Walrus operator (`:=`) or complex one-line f-string logic. Write clear, multi-line, readable code.
4. **No Fragmentation:** Deliver the entire logic as one single, cohesive, runnable `.py` file.

**Deliverables:**

1. **The Python Script:** A single, clean, well-commented `.py` file.
2. **The Setup Guide:** Instructions for `venv` creation, dependency installation (`pip install requests`), and execution.

---

## The Key Insight

The self-improving loop works because:

1. **The error is concrete.** Not "it's broken," but "line 42 has `encoding='cap=utf-8'` which is invalid Python."
2. **The model has full context.** It sees the original prompt, the code it generated, and the exact error.
3. **Constraints accumulate.** Each iteration adds explicit rules that prevent past failures. The prompt gets longer but more reliable.
4. **The human is still in the loop.** You're not asking the model to evaluate itself — you're giving it specific, factual feedback.

## What You've Learned

- The rerun pattern feeds errors back to the model with full context
- The master prompt (plan4) is the result of 4 rounds of iterative improvement
- Defensive constraints (anti-hallucination rules) prevent specific failure modes
- A good error message is the difference between "try again" and "fix this exact thing"

**Next: [08 — Code Extraction →](/run-ai-locally/03-automation/08-code-extraction/)**