IRCP Training Module

IRCP Training Module

This module provides comprehensive training functionality for the Enhanced Inverse Ring Contextual Propagation (IRCP) framework.

Features

ICPTrainer Class

The `ICPTrainer` class implements a sophisticated training pipeline with the following capabilities:

#### Multi-Component Loss Function
1. Coordinate Prediction Loss: MSE loss for predicting 4D coordinates (x, y, z, t)
2. Embedding Consistency Loss: Ensures similar coordinates have similar embeddings
3. Conservation Constraint Loss: Implements measure preservation constraints
4. Topological Consistency Loss: Preserves local neighborhood structure
5. L2 Regularization: Prevents overfitting

#### Advanced Training Features
- Adaptive Learning Rate Scheduling: Cosine, step, and exponential schedulers
- Gradient Clipping: Prevents gradient explosion
- Comprehensive Logging: Detailed loss component tracking
- Checkpoint Management: Automatic saving of best models
- Progress Visualization: Real-time training progress with component losses

#### Model Compatibility
- PyTorch Models: Standard `forward()` method support
- Custom ICP Models: `predict_coordinates()` method support
- Fallback Architecture: Automatic linear layer creation for unsupported models

#### Training Statistics
- Real-time Metrics: Loss components, learning rates, gradient norms
- Model Analysis: Parameter counts, memory usage, efficiency metrics
- Training Plots: Automatic generation of loss curves and progress charts

Usage

Basic Training

from ircp.training import ICPTrainer

# Initialize trainer
trainer = ICPTrainer(model, config)

# Train model
results = trainer.train(train_data, val_data)

Advanced Configuration

config = {
    "epochs": 100,
    "batch_size": 32,
    "learning_rate": 1e-4,
    "optimizer": "adamw",
    "scheduler": "cosine",

    # Loss weights
    "coord_loss_weight": 1.0,
    "consistency_loss_weight": 0.1,
    "conservation_loss_weight": 0.05,
    "topological_loss_weight": 0.1,
    "l2_reg_weight": 1e-5,

    # Training settings
    "max_grad_norm": 1.0,
    "save_checkpoints": True,
    "save_interval": 10,
    "output_dir": "./checkpoints"
}

trainer = ICPTrainer(model, config)

Model Export

# Export in different formats
trainer.export_model("model.pt", format="pytorch")
trainer.export_model("model.onnx", format="onnx")
trainer.export_model("model_script.pt", format="torchscript")

Training Analysis

# Get comprehensive statistics
stats = trainer.get_training_statistics()

# Plot training progress
trainer.plot_training_progress("training_plots.png")

# Validate model architecture
validation = trainer.validate_model_architecture()

Loss Components

1. Coordinate Prediction Loss

coord_loss = MSELoss()(predicted_coords, true_coords)

### 2. Embedding Consistency Loss
Ensures that points with similar coordinates have similar embeddings:

coord_dists = torch.cdist(coordinates, coordinates, p=2)
embedding_sims = torch.mm(embeddings_norm, embeddings_norm.t())
consistency_loss = MSELoss()(1 - embedding_sims, coord_dists / coord_dists.max())

### 3. Conservation Constraint Loss
Implements measure preservation using covariance determinants:

coord_cov = torch.mm(coord_centered.t(), coord_centered) / (n - 1)
embed_cov = torch.mm(embed_centered.t(), embed_centered) / (n - 1)
conservation_loss = abs(log(det(embed_cov)) - log(det(coord_cov)))

### 4. Topological Consistency Loss
Preserves k-nearest neighbor relationships:

# Find k-NN in coordinate space
coord_knn = topk(coord_distances, k)
# Ensure same relationships in embedding space
topo_loss = MSELoss()(embed_neighbor_dists, coord_neighbor_dists)

Configuration Options

Output Files

The trainer generates several output files:

• `checkpoint_epoch_N.pt`: Regular training checkpoints
• `best_model.pt`: Best model based on validation loss
• `training_history.json`: Complete training metrics history
• `training_progress.png`: Training progress visualization
• `model_stats.json`: Model architecture and parameter statistics

Integration with IRCP Framework

The trainer integrates seamlessly with the main IRCP framework:

from ircp import ICPFramework

framework = ICPFramework(database_path="conversations.db")
framework.load_conversations()
framework.initialize_model()

# Training uses the ICPTrainer automatically
results = framework.train(epochs=50, batch_size=32)