Validation Scripts & Edge Cases

The Problem: Manual Checking Doesn’t Scale

After classifying 5 files, you can manually verify each one. After classifying 50, you need tools. After classifying 210, you must have automated validation.

The classification agent does validation as part of its workflow, but that’s slow and wasteful — the agent shouldn’t be checking its own work line by line. Extract validation into a dedicated script.

validate_taxonomy.py

# Validate a single file
mise exec -- python validate_taxonomy.py content/decorated/FILE.md \
    --source-dir content/uncategorized/ --lenient

# Validate all decorated files
mise exec -- python validate_taxonomy.py content/decorated/ \
    --source-dir content/uncategorized/ --lenient

What It Checks

Edge Cases It Handles

Unicode normalization: Fancy quotes (Unicode) vs ASCII quotes. The --lenient flag normalizes Unicode before comparison:

# Without --lenient: "Karpathy's" != "Karpathy's" (different quote characters)
# With --lenient: both normalized to ASCII before comparison

Malformed source YAML: Some source files have unescaped quotes in their original_file field. The validator catches this and reports it separately — the decorated file is correct, the source file has the issue.

Missing files: An EML file might exist but its markdown wasn’t generated. A decorated file might exist but its source was deleted.

classify_emails.py — Progress Tracking

mise exec -- python classify_emails.py --status
# Output:
# Total: 210
# Classified: 66 (31.4%)
# Remaining: 144

mise exec -- python classify_emails.py --list     # List unclassified
mise exec -- python classify_emails.py --next 5    # Get next 5 to classify
mise exec -- python classify_emails.py --validate  # Validate all decorated

This script is the dashboard for the pipeline. It answers:

• How many files are left to process?
• Which files should be classified next?
• Are all decorated files valid?

check_emails.py — Completeness Check

mise exec -- python check_emails.py --status
# Output:
# Total EML files:    218
# Converted:          146
# Not converted:      72
# Progress:           67.0%

# List unconverted files
mise exec -- python check_emails.py

# Get next 5 to convert (oldest first)
mise exec -- python check_emails.py --next 5

This script parses original EML files, extracts dates and subjects, generates the expected markdown filename, and checks if it exists. It catches:

• Missed conversions: EML files that were never run through the converter
• Filename mismatches: The converter and checker must use the same filename generation logic
• Incomplete batches: Human error in selecting which files to convert

The Validation Pipeline

EML files (emails/)         Markdown files (uncategorized/)     Decorated files (decorated/)
        │                              │                                │
        └── check_emails.py ───────────┘                                │
        checks each .eml has                                           │
        a corresponding .md                                             │
                                                                        │
                                      └── classify_emails.py ───────────┘
                                      tracks which files are
                                      classified vs pending
                                                                        │
                                                                        └── validate_taxonomy.py
                                                                        checks structure and content
                                                                        of every decorated file

All scripts → single source of truth for "what state is the pipeline in?"

Real-World Validation Results

After the complete run:

Email Processing Complete
- Converted: 218/218 EML files (100%)
- Classified: 210/210 markdown files (100%)
- Validated: 207/210 files pass validation (3 have minor issues)

The 3 validation errors are likely due to source file YAML issues
(like unescaped quotes in the original_file field), not problems
with the decorated output files themselves.

207 out of 210 pass — 98.6% success rate. The 3 failures aren’t classification errors; they’re source file issues that the validator correctly flags for manual review.

Why Validation Matters

Without validation, you’d have:

• Silent errors: Bad YAML that breaks parsers downstream
• Inconsistent metadata: Some files missing audience, others with typos in series
• Truncated content: Body text cut off by a parsing bug
• Drift: The classification agent’s output quality would degrade without feedback

Validation closes the feedback loop:

Classify → Validate → Fix → Reclassify → Validate → Done

Each validation failure is a signal: either the agent made a mistake, or the source data has an issue. Either way, you now know about it.

The Meta-Pattern

All three scripts — validate_taxonomy.py, classify_emails.py, check_emails.py — were built by the AI in response to observations:

1. “The agent is doing redundant checking” → Extract into script
2. “How much progress have we made?” → Build status tracker
3. “Are we missing any emails?” → Build completeness checker

You don’t anticipate every tool you’ll need at the start. You build them as you discover needs. This is iterative tooling — each script solves a specific problem that became visible during the process.

Next: Iterate & Improve — Refine the schema, re-run classification, and build a magazine website.