SEA Re-Embedding Complete — Topic-Augmented Embeddings

SEA Re-Embedding Complete — Topic-Augmented Embeddings

Summary

Re-embedded all 13 skill entities with topic-augmented text for improved Tier 1 matching. The core problem was that embeddings were generated from raw SKILL.md content (technique descriptions, markdown formatting) which sits in a different semantic space than user queries. Fix: restructure embedding text to be query-dominant.

What Changed

1. `embedding_indexer.py` — Topic-Augmented `_extract_skill_text()`

Before: `"{skill_name}: {yaml_description}skill_md_body"` truncated to 800 chars. The body was SKILL.md markdown — technique instructions, not user query language.

After: Query-dominant structure (~70

New components:
- `_load_skill_topics()` — reads `hot_topics` and `cold_topics` from each skill's `state.json`
- `SKILL_EXAMPLE_QUERIES` — 7-10 natural-language user queries per skill that should trigger that skill
- SKILL.md body stripped — only the YAML `description:` field is kept (body was semantic noise)
- Budget increased from 800 to 1200 chars to accommodate richer text

2. `sea_skill_injector.py` — Keyword Pattern Parity

Before: `get_sea_skill_patterns()` (lines 397-411, used by spawner.py) was missing patterns that existed in `_keyword_fallback()` (lines 153-167):

After: Both functions now have identical pattern sets.

3. `embedding-cache.npz` — Rebuilt

New 13x384 cache at `[home-path]` with topic-augmented vectors.

Test Results

Custom Query Test (10 queries, threshold 0.4)

**Hit rate: 9/10 (90

*phi:veritas limitation: "hype" dominates the semantic space in all-minilm, pulling the vector toward marketing/social rather than truth-verification. phi:veritas IS the top-ranked skill for this query (0.26), but below threshold. The `_keyword_fallback` catches `\btruth\b` and `\bverif` for these cases.

Built-in Tier 1 Router Tests

12/12 passed

All expect-high queries match their target skills, both expect-low queries (groceries, timezone) score 0.000.

Before vs After Comparison

Files Changed

• `[home-path]` (modified) — added `_load_skill_topics()`, `SKILL_EXAMPLE_QUERIES`, restructured `_extract_skill_text()`
• `[home-path]` (modified) — synced `get_sea_skill_patterns()` with `_keyword_fallback()`
• `[home-path]` (rebuilt) — new 13x384 topic-augmented cache

Chunked Evil Flow Report

Chunk: Embedding text augmentation | Status: ✅ | Evil findings: 1 | Fixes: 1 | Gate: pass
  Finding: SKILL.md body noise dilutes embedding signal → Fix: strip body, keep only description

Chunk: Keyword pattern sync | Status: ✅ | Evil findings: 1 | Fixes: 1 | Gate: pass
  Finding: 6 patterns missing from get_sea_skill_patterns → Fix: synced both dicts

Chunk: Cache rebuild & validation | Status: ✅ | Evil findings: 1 | Fixes: 0 | Gate: pass
  Finding: phi:veritas <0.4 for colloquial "hype" queries — model limitation, not fixable without upgrading from all-minilm. Keyword fallback covers this case.

Synthesis: Integration seams intact. No state leakage. Embedding cache format unchanged (npz with same keys). Hot-reload mechanism will pick up new cache via mtime check.

Recommendations

1. Model upgrade: Consider upgrading from `all-minilm` (384-dim) to `nomic-embed-text` (768-dim) for better abstract/philosophical concept matching. Would likely push phi:veritas above 0.4 threshold.
2. Dynamic query learning: As activation logs accumulate, extract actual user queries that triggered skills and add them to `SKILL_EXAMPLE_QUERIES` for iterative embedding improvement.
3. Hybrid scoring: For the 0.25-0.40 "gray zone", combine Tier 1 embedding score with keyword match count for a blended score before applying threshold.