Project Ideas

The patterns in this course apply far beyond email newsletters. Here are ideas for applying them to your own work.

The Patterns, Generalised

Content Processing Projects

Migrate a Documentation Site

• Source: Old docs in various formats (MediaWiki, Confluence, Word docs)
• Process: Extract → convert to markdown → classify by topic/audience → cross-link
• Patterns: Extraction, taxonomy design, semantic enrichment

Organize a Personal Knowledge Base

• Source: Years of notes, bookmarks, saved articles
• Process: Categorize by domain, extract key concepts, link related items
• Patterns: Classification, parallel processing, validation

Build a Company Knowledge Graph

• Source: Internal wikis, meeting notes, project docs, emails
• Process: Extract entities (people, projects, technologies), link by relationships
• Patterns: Semantic enrichment, taxonomy design, cross-linking

Curate a Research Paper Library

• Source: Downloaded PDFs, arXiv references, citation lists
• Process: Extract paper metadata, classify by field, identify citation networks
• Patterns: Extraction, classification, entity extraction

Process Customer Feedback

• Source: Survey responses, support tickets, reviews
• Process: Categorize by topic/sentiment/urgency, extract mentioned features
• Patterns: Classification, parallel processing, semantic enrichment

Archive and Index a Mailing List

• Source: Years of mailing list archives (mbox, EML)
• Process: Extract threads, identify key contributors, categorize discussions
• Patterns: Extraction (same EML pipeline!), classification, people extraction

Agent Patterns to Try

Hierarchical Classification

Instead of one pass, use a multi-stage classification:

Stage 1: Rough category (technical vs non-technical)
    ↓
Stage 2: Subcategory (language models, infrastructure, ethics)
    ↓
Stage 3: Detailed tags and entities

Each stage uses a different agent optimized for that level of granularity.

Ensemble Classification

Run multiple classification agents on the same content and compare results. Disagreements highlight ambiguous cases that need human review.

Review-and-Refine Loop

Classify → Review agent checks output → Flag issues → Re-classify flagged items

The review agent catches errors that the classification agent misses, creating a self-correcting loop.

Incremental Classification

Process new content as it arrives rather than in batches. The classification agent runs on each new item, using existing data for context:

New file → Classify → Compare with existing → Flag outliers → Add to collection

Multi-Model Enrichment

Use different models for different enrichment tasks:

Build Your Own Pipeline

A generic content pipeline skeleton:

source-files/
    ↓
1. Research phase
    - Agent analyzes sample → recommends approach
    - Plan written to plans/
    ↓
2. Extraction phase
    - Converter script (built by agent)
    - Output: content/raw/
    ↓
3. Taxonomy phase
    - Agent reads all content → proposes schema
    - Schema saved to reports/
    ↓
4. Classification phase
    - Classification agent (built by agent)
    - Parallel processing: content/enriched/
    ↓
5. Validation phase
    - Validation scripts (built by agent)
    - Progress tracker (built by agent)
    ↓
6. Application phase
    - Generate JSON data
    - Build website/search/dashboard

Pro Tips

1. Start with a small sample (5-10 files) to validate your approach before scaling to hundreds
2. Write CLAUDE.md early — it compounds across sessions
3. Save all research to reports/ — don’t let findings live only in chat history
4. Use the plain text version of emails when available — often already markdown
5. Make agents stateless — design for parallelism from the start
6. Build validation scripts incrementally — extract them when you see redundant work
7. Re-run the full pipeline after schema changes — don’t patch individually

Back to Course Overview | Start with Phase 1: Setting Up