Grand Diomande Research · Full HTML Reader

DLM-RCP Integration

- **Cross-Conversation Understanding**: Query across all 277 conversations simultaneously - **Unified Knowledge Base**: Treat all conversations as one interconnected system - **Dynamic Context Assembly**: Automatically find and assemble relevant messages - **Knowledge Evolution**: Track knowledge building without regression - **High Performance**: Built-in caching and optimization

Agents That Account for Themselves proposal experiment writeup candidate score 24 .md

Full Public Reader

DLM-RCP Integration

This module provides integration between the Dynamic Language Model (DLM) and Ring Contextual Propagation (RCP) systems.

Overview

The RCP Bridge enables DLM to leverage RCP's cross-conversation understanding capabilities:

  • Cross-Conversation Understanding: Query across all 277 conversations simultaneously
  • Unified Knowledge Base: Treat all conversations as one interconnected system
  • Dynamic Context Assembly: Automatically find and assemble relevant messages
  • Knowledge Evolution: Track knowledge building without regression
  • High Performance: Built-in caching and optimization

Quick Start

Basic Usage

python
from dlm.integration.rcp_bridge import create_rcp_bridge

# Create and initialize bridge
bridge = create_rcp_bridge(
    database_path="path/to/conversations.db",
    auto_initialize=True,
    verbose=True
)

# Query with RCP
result = bridge.query_with_rcp("How does authentication work?")

print(f"Response ID: {result.response_id}")
print(f"Confidence: {result.confidence:.3f}")
print(f"Sources: {len(result.source_conversations)} conversations")
print(f"Messages: {result.message_count}")
print(f"\nContext:\n{result.assembled_context}")

Advanced Usage

python
from dlm.integration.rcp_bridge import RCPBridge

# Initialize bridge manually
bridge = RCPBridge(database_path="conversations.db")
bridge.initialize(verbose=True)

# Query with custom parameters
result = bridge.query_with_rcp(
    query="Explain database optimization",
    max_context_messages=100,
    use_cache=True
)

# Expand context if needed
if result.is_expandable:
    expanded = bridge.expand_context(
        response_id=result.response_id,
        additional_messages=30
    )
    print(f"Expanded to {expanded.message_count} messages")

# Get conversation-specific context
conv_context = bridge.get_conversation_context(
    conversation_id="conv_12345",
    include_cross_conversation=True
)

# Find similar messages
similar = bridge.find_similar_messages(
    message_id="msg_67890",
    max_similar=20
)

# Check system status
status = bridge.get_system_status()
print(f"Queries processed: {status['bridge_stats']['queries_processed']}")
print(f"Cache hit rate: {status['bridge_stats']['cache_hit_rate']:.2%}")

Features

RCPBridge Class

Main bridge class for DLM-RCP integration.

Methods:

  • `initialize(verbose=True)` - Initialize RCP system (loads all conversations)
  • `query_with_rcp(query, max_context_messages=50, use_cache=True)` - Query with cross-conversation understanding
  • `expand_context(response_id, additional_messages=20)` - Expand previous response with more messages
  • `get_conversation_context(conversation_id, include_cross_conversation=True)` - Get conversation context
  • `find_similar_messages(message_id, max_similar=20)` - Find similar messages across conversations
  • `get_system_status()` - Get system status and statistics
  • `clear_cache()` - Clear query cache
  • `export_knowledge_state(output_path)` - Export knowledge state for backup

RCPQueryResult Class

Result object from RCP queries.

Attributes:

  • `query` (str) - Original query
  • `response_id` (str) - Unique response identifier
  • `assembled_context` (str) - Assembled text context from relevant messages
  • `raw_context` (DynamicContext) - Full RCP context object
  • `confidence` (float) - Response confidence score [0, 1]
  • `source_conversations` (List[str]) - IDs of source conversations
  • `knowledge_clusters` (List[int]) - Knowledge clusters used
  • `message_count` (int) - Number of messages in context
  • `temporal_span` (Tuple[float, float]) - Time range of messages
  • `knowledge_state_id` (str) - Current knowledge state ID
  • `knowledge_gain` (Optional[float]) - Knowledge gain from query
  • `is_expandable` (bool) - Whether context can be expanded
  • `processing_time` (float) - Query processing time in seconds
  • `timestamp` (float) - Result timestamp

Methods:

- `to_dict()` - Convert to dictionary for API responses

Performance

Caching

The bridge includes built-in query caching:

  • Cache Size: Up to 1000 queries (LRU eviction)
  • Cache Key: Exact query string
  • Benefits: Instant responses for repeated queries

Statistics

Track bridge performance:

python
status = bridge.get_system_status()
stats = status['bridge_stats']

print(f"Queries: {stats['queries_processed']}")
print(f"Cache hits: {stats['cache_hits']}")
print(f"Hit rate: {stats['cache_hit_rate']:.2%}")
print(f"Avg confidence: {stats['avg_confidence']:.3f}")
print(f"Avg time: {stats['avg_processing_time']:.3f}s")

Integration with DLM

In DLM Inference Pipeline

python
from dlm.inference.pipeline import InferencePipeline
from dlm.integration.rcp_bridge import RCPBridge

class RCPEnhancedPipeline(InferencePipeline):
    def __init__(self, database_path, *args, **kwargs):
        super().__init__(*args, **kwargs)
        self.rcp_bridge = RCPBridge(database_path)
        self.rcp_bridge.initialize()

    def generate_response(self, query):
        # Get RCP context
        rcp_result = self.rcp_bridge.query_with_rcp(query)

        # Use RCP context in generation
        context = rcp_result.assembled_context
        confidence = rcp_result.confidence

        # Generate with enhanced context
        response = self.generate_with_context(query, context)

        return response, confidence

As API Endpoint

python
from fastapi import FastAPI
from dlm.integration.rcp_bridge import create_rcp_bridge

app = FastAPI()
bridge = create_rcp_bridge("conversations.db", auto_initialize=True)

@app.post("/query_rcp")
async def query_rcp(query: str):
    result = bridge.query_with_rcp(query)
    return result.to_dict()

@app.get("/system_status")
async def system_status():
    return bridge.get_system_status()

Architecture

┌─────────────────────────────────────────────────────┐
│                   DLM System                        │
│                                                     │
│  ┌──────────────┐         ┌──────────────┐        │
│  │   Inference  │         │  Generation  │        │
│  │   Pipeline   │         │   Engine     │        │
│  └──────┬───────┘         └──────┬───────┘        │
│         │                        │                 │
│         └────────┬───────────────┘                 │
│                  │                                 │
│         ┌────────▼────────┐                        │
│         │   RCP Bridge    │  ◄── Integration Layer│
│         └────────┬────────┘                        │
└──────────────────┼──────────────────────────────────┘
                   │
         ┌─────────▼─────────┐
         │   RCP System      │
         │                   │
         │ ┌───────────────┐ │
         │ │ Unified       │ │
         │ │ Knowledge     │ │
         │ │ System        │ │
         │ └───────┬───────┘ │
         │         │         │
         │ ┌───────▼───────┐ │
         │ │ Dynamic       │ │
         │ │ Context       │ │
         │ │ Builder       │ │
         │ └───────┬───────┘ │
         │         │         │
         │ ┌───────▼───────┐ │
         │ │ Knowledge     │ │
         │ │ Evolution     │ │
         │ │ Engine        │ │
         │ └───────────────┘ │
         └───────────────────┘

Error Handling

python
from dlm.integration.rcp_bridge import RCPBridge

bridge = RCPBridge("conversations.db")

try:
    # Must initialize before use
    result = bridge.query_with_rcp("test")
except RuntimeError as e:
    print(f"Error: {e}")  # "RCP system not initialized"

# Proper usage
bridge.initialize()
result = bridge.query_with_rcp("test")  # Works!

Testing

See `test_rcp_bridge.py` for comprehensive tests:

bash
cd packages/dlm/integration
python test_rcp_bridge.py

Troubleshooting

Import Errors

If you see import errors related to DLM's main `__init__.py`, import directly:

python
# Instead of:
from dlm.integration import RCPBridge  # May fail

# Use:
from dlm.integration.rcp_bridge import RCPBridge  # Always works

Memory Usage

For large databases (>100k messages), initialization may take 5-10 minutes and use significant memory. Consider:

  • Running initialization once and caching the result
  • Using a separate service/process for RCP
  • Filtering conversations if full history isn't needed

Performance

If queries are slow:

  • Enable caching (enabled by default)
  • Reduce `max_context_messages` parameter
  • Use `verbose=False` during initialization
  • Check database indices are optimized

Examples

See `examples/` directory for:

  • `basic_usage.py` - Simple query examples
  • `advanced_integration.py` - Full DLM integration
  • `performance_test.py` - Performance benchmarking
  • `api_server.py` - FastAPI REST API

License

Same as parent DLM project.

Support

For issues or questions:
1. Check this README
2. See `docs/RCP_PRODUCTION_ROADMAP.md`
3. Review RCP documentation in `packages/rcp/`

Promotion Decision

Attach run IDs, datasets, metrics, and reproduction commands.

Source Anchor

Comp-Core/backend/cc-trajectory/legacy/cc-tpo-original/cc-tpo/packages/dlm/integration/README.md

Detected Structure

Method · Evaluation · Code Anchors · Architecture