--- title: "Semantic Enrichment" description: LLMs extract people, companies, models, audience level, and content depth — metadata that would be tedious to create manually. --- # Semantic Enrichment ## What Semantic Enrichment Means The decorated files contain metadata that wasn't in the original emails. This metadata — who's mentioned, which companies appear, what models are discussed, who the audience is, how deep the content goes — is **extracted by the LLM** from the content itself. This is the **Semantic Enrichment** pattern: using LLMs to add structured meaning to unstructured content. It's one of the most valuable things AI agents can do. ## What Gets Extracted ### People (with Roles and Affiliations) ```yaml people_mentioned: - name: "Andrej Karpathy" affiliation: "OpenAI (former)" - name: "Andrew Ng" affiliation: "Landing AI" - name: "Fei-Fei Li" affiliation: "Stanford" - name: "Holden Karau" # No affiliation listed — agent can leave it empty ``` The agent identifies people from context: names in attributions, quoted speakers, people referenced in article text. It often captures roles and affiliations too, even though those aren't always explicitly stated in the same sentence. ### Companies ```yaml companies_mentioned: [ "Cursor", "Hugging Face", "OpenAI", "Anthropic", "Meta", "Google", "Microsoft", "Aleph Alpha", ] ``` Notice this includes companies of all sizes — from major labs (OpenAI, Google) to startups (Cursor, Aleph Alpha). The agent doesn't need a predefined list; it recognizes company names in context. ### AI Models ```yaml models_mentioned: ["GPT-4o", "Pharia-1-LLM", "Phi-3.5", "Hermes 3", "Jamba-1.5", "Gemini"] ``` Models are often mentioned in passing — "OpenAI's GPT-4o," "Meta released Llama 3," "a new paper benchmarks DeepSeek-V3." The agent extracts them from the surrounding text, handling variations in naming conventions. ### Content Characteristics ```yaml audience: mixed depth: overview has_research_papers: true has_code_examples: false is_premium: false ``` These are subjective judgments that the LLM is surprisingly good at: - **Audience**: Is this written for ML researchers (`technical`), business leaders (`business`), general readers (`general`), or a mix (`mixed`)? - **Depth**: Is this a high-level summary (`overview`), a moderately detailed piece (`intermediate`), or a comprehensive deep-dive (`deep-dive`)? - **Research papers**: Does the content reference specific academic papers? - **Code examples**: Are there code snippets in the content? - **Premium**: Is this premium subscriber content? ### Tags ```yaml tags: [ "cursor", "lerobot", "indie-hacking", "robotics", "fine-tuning", "ai-assistants", "democratization", ] ``` Tags are freeform — the agent identifies specific technologies, techniques, and concepts discussed in the article. These aren't from a controlled vocabulary; the agent generates them from the content. ## How the Agent Does It The classification agent reads the entire markdown file (often 2-8KB of text) and: 1. **Identifies the series** from the subject line (regex + emoji detection) 2. **Determines content type** from structure (multiple sections = digest, Q&A format = interview, etc.) 3. **Assigns a primary topic** based on overall theme 4. **Extracts entities** (people, companies, models) by scanning for known patterns and context 5. **Judges audience and depth** from writing style and technical level 6. **Generates tags** from key concepts discussed 7. **Sets boolean flags** (has papers, has code, is premium) from content inspection All of this happens in a single pass. The agent doesn't need multiple API calls or a pipeline of specialized extractors — one LLM call per file handles everything. ## The Power of This Pattern ### Scale Doing this manually for 210 files at even 5 minutes each would take 17.5 hours. The agent does it in ~15 minutes (a 70x speedup). ### Consistency Humans get tired, inconsistent, or biased after classifying dozens of items. The agent applies the same taxonomy rules to every file. The 3 validation failures were due to malformed source YAML, not classification errors. ### Richness A human classifier might tag a few topics and call it done. The agent reliably extracts: - 430 unique people with affiliations - 553 unique companies - 615 unique model mentions - Hundreds of content-specific tags This creates a **cross-reference graph** — from any article, you can navigate to every person, company, and model it mentions. ### Discoverability Before classification, finding content meant scanning filenames or full-text search. After classification, you can query: - "Show me all deep-dives about agents from 2024" - "Which articles mention Andrej Karpathy?" - "What are the top companies discussed in FOD digests?" - "List all technical explainers mentioning Llama models" - "Show interview transcripts with audience=mixed" The frontmatter becomes a queryable database. ## The Cross-Reference Network The extracted metadata enables rich cross-linking: ``` Article: "Golden Age for Indie Devs" mentions → Andrej Karpathy, Andrew Ng mentions → Cursor, Hugging Face, OpenAI mentions → GPT-4o, Phi-3.5, Hermes 3 ↓ Entity page: "Andrej Karpathy" appears in → 34 articles ← links to → OpenAI, Tesla ← connected to → Andrew Ng, Fei-Fei Li ↓ Entity page: "OpenAI" appears in → 133 articles ← connected to → Anthropic, Google, Microsoft ``` This is possible because the agent consistently extracts and normalizes entity names across all files. ## Why LLMs Are Uniquely Good at This Traditional NLP approaches to entity extraction (spaCy, NER models) can identify people and organizations but can't: - **Disambiguate context**: Is "Anthropic" a company or an adjective? - **Recognize novel entities**: A new startup mentioned for the first time - **Identify AI model names**: "GPT-4o" or "Pharia-1-LLM" aren't in standard NER training data - **Judge content characteristics**: Audience level and depth require reading comprehension, not pattern matching - **Handle varied formats**: Podcast transcripts, digests, explainers, and profiles all have different structures LLMs handle all of these naturally. They understand that "Cursor" in an AI newsletter is a company (not a UI element) and that "Hermes 3" is a model release (not a Greek god). --- **Next: [Phase 4: Validation](/content-repurpose/04-validation/)** — Scripts, edge cases, and iterative refinement.