Graph-Augmented Recursive Language Models for Personal Knowledge Systems

We present Cog-RLM, a graph-augmented recursive language model architecture for personal knowledge systems that achieves 90.3% accuracy on a comprehensive 103-question multi-dimensional evaluation using a stock 3-billion parameter model with zero fine-tuning and zero inference cost. Our system extends the Recursive Language Model (RLM) paradigm (Zhang et al., 2025) with three novel contributions: (1) a local knowledge graph providing relationship-aware context retrieval, (2) a hybrid decomposition classifier that selectively triggers recursive processing only for multi-hop queries, and (3) a persistent semantic retrieval layer using sentence-transformer embeddings. We evaluate across ten dimensions grouped into four categories—Retrieval (recall, precision, consistency), Reasoning (reasoning, inference), Robustness (counterfactual, adversarial, negation), and Flexibility (temporal, generalization)—demonstrating that architectural augmentation of small models can match or exceed the performance of larger fine-tuned models on domain-specific knowledge tasks. The full evaluation suite comprises 174 regression tests including behavioral policy compliance and format adherence checks. Our approach requires no training data, runs entirely on consumer hardware (Apple M4, 16GB), and processes queries in 1.0–12.5 seconds. **Keywords:** Recursive Language Models, Knowledge Graphs, Retrieval-Augmented Generation, Personal AI, Small Language Models