---
title: "Chapter 03: Tests That Push Back"
description: "Run a deterministic application suite and prove the gate catches a deliberate defect."
type: "lesson"
order: 3
chapter: "01-reproducible-foundations"
duration: "45–90 minutes"
---

# Chapter 03: Tests That Push Back

## Objective

Run a deterministic application suite and prove the gate catches a deliberate defect.

Your tutor should stop at each gate and wait for your evidence before continuing.

## Concept

A useful test makes the desired behavior observable without asking the agent whether it succeeded. Our task board stores dictionaries with a title and a completion flag. Whitespace normalization, title length, and pending-item ordering are small enough to test exhaustively at their boundaries.

The starter implements existing behavior. A later exercise adds task completion. Begin with six passing tests so you can separate a broken environment from a missing feature. A red test is useful only when you know why it is red.

`devenv test` runs the configured test entry point; it does not magically discover every test in your repository. We explicitly wire formatting, lint, and unit tests into it. The same command becomes the human, agent, and CI acceptance gate.

## Concept check

**Q1.** Why deliberately introduce a bug before giving the project to an agent?

**Q2.** What happens if your test entry point only prints “passed”?

<details>
<summary>Concept answer key — attempt first</summary>

## Answer key (concept)

**Q1.** It proves the check rejects an incorrect implementation. **Pass criteria:** explain a negative control.

**Q2.** The command can succeed without testing behavior. **Pass criteria:** exit status is meaningful only when backed by real checks.

</details>

## Gate: Prepare source and tests

Copy `taskboard.py`, `tests/`, `exercises/`, and `scripts/` from the archive into your project. Merge the archive’s `.gitignore` rules before generating Python bytecode or logs. Do not copy `exercises/test_completion.py` into `tests/` yet.

Read the implementation:

```python
"""A deliberately small task board for the Nix agent course."""


def normalize_title(value: str) -> str:
    if not isinstance(value, str):
        raise TypeError("title must be text")
    title = " ".join(value.split())
    if not title or len(title) > 80:
        raise ValueError("title must contain 1 to 80 characters")
    return title


def new_task(title: str) -> dict:
    return {"title": normalize_title(title), "done": False}


def pending_titles(tasks: list[dict]) -> list[str]:
    return [task["title"] for task in tasks if not task["done"]]
```

The test suite covers blank strings, an 80/81-character boundary, non-text input, new tasks, whitespace, and order. None of those checks uses an LLM or a network service.

## Gate: Smoke-test the application

```bash
devenv shell python -m unittest discover -s tests -v
```

**Pass:** six tests, no skipped checks, exit status zero. Run this before adding services.

## Gate: Make one acceptance command

Replace your initial `devenv.nix` with the archive version:

```nix
{ pkgs, ... }:
{
  packages = with pkgs; [ git jq ripgrep curl bashInteractive ruff ];
  languages.python.enable = true;

  tasks."check:format".exec = "ruff format --check taskboard.py tests exercises scripts";
  tasks."check:lint".exec = "ruff check taskboard.py tests exercises scripts";
  tasks."check:unit".exec = "python -m unittest discover -s tests -v";
  tasks."check:all".after = [ "check:format" "check:lint" "check:unit" ];

  enterTest = ''
    set -euo pipefail
    ruff format --check taskboard.py tests exercises scripts
    ruff check taskboard.py tests exercises scripts
    python -m unittest discover -s tests -v
  '';
}
```

```bash
devenv test
```

The task declarations are explained in Chapter 05. For now, `enterTest` is the authoritative sequential gate. `set -euo pipefail` prevents an earlier failing command from being hidden by a later successful command.

## Gate: Prove red, then green

Change the `> 80` title check to `> 79`. Run the unit suite and then `devenv test`. Both should fail at the boundary case. Restore `> 80`, run both again, and record the observed failure and recovery.

Commit this known-good baseline. Do not merely report that tests “probably catch it.” Show your tutor the failed assertion and passing rerun.

## Gate: Improve the evidence

Add a test for a title containing Unicode characters. Decide whether length means Python characters, bytes, or user-perceived graphemes; document the chosen contract. This lab uses Python’s `len`, not display width. Review the test before running an agent.

## Known failure modes

- Zero tests is not a pass: verify the reported count.
- Running from the wrong directory can cause `ModuleNotFoundError`; run from the lab root.
- Formatting checks reject style but do not establish correctness.
- A test that asserts the same implementation expression can share the same mistake. Assert a concrete outcome from a spec.

## Check your understanding

**Q3.** What does the 80/81 pair establish?

**Q4.** Which test should catch changing the order of pending tasks?

<details>
<summary>Answer key — attempt every question first</summary>

## Answer key

**Q3.** The exact inclusive length boundary. **Pass criteria:** both valid and invalid neighbors.

**Q4.** The pending-order test with distinct titles. **Pass criteria:** identify observable ordering, not implementation details.

</details>

## References

- [devenv tests](https://devenv.sh/tests/)
- [Python unittest](https://docs.python.org/3/library/unittest.html)

---

[← Chapter 02](/nix-agent-lab/01-reproducible-foundations/02-lock-the-environment/) · [Next: Chapter 04 →](/nix-agent-lab/02-services-and-contracts/04-real-services/)