Comp-Core Architecture Overview
**Version**: 2.4.0 **Last Updated**: 2026-01-03 **Schema Version**: 1.0.0 **Implementation Status**: Production Ready **Maintainer**: Computational Choreography Team
Full Public Reader
Comp-Core Architecture Overview
Version: 2.4.0
Last Updated: 2026-01-03
Schema Version: 1.0.0
Implementation Status: Production Ready
Maintainer: Computational Choreography Team
---
Implementation Status Matrix
| Component | Status | Location | Tests | Deployment | Notes |
|---|---|---|---|---|---|
| cc-relay (Rust) | ✅ Production | `core/cc-relay` | 11 pass | DO VPS | TLV parser, circuit breaker, Prometheus |
| cc-protocol | ✅ Production | `core/cc-protocol` | 176 pass | Crate | 27-bone skeleton, MessagePack serialization |
| cc-collection | ✅ Production | `core/cc-collection` | 24 pass | Crate | EKF sensor fusion, occlusion handling |
| cc-anticipation | ✅ Production | `core/cc-anticipation` | 50 pass | Crate | Commitment/uncertainty kernel |
| cc-gesture | ✅ Production | `core/cc-gesture` | 81 pass | Crate | Neighbor voting classifier |
| DELL | ✅ Production | `core/cc-echelon/crates/dell` | 343 pass | GCP VM | Dual-equilibrium latent learning |
| cc-conductor | ✅ Production | `core/cc-echelon/crates/cc-conductor` | 31 pass | Crate | Beat-quantized scheduling |
| cc-mcs-headless | ✅ Production | `backend/cc-mcs/src-tauri` | - | GCP VM | Headless daemon |
| RAG++ Core | ✅ Production | `core/cc-rag-plus-plus/crates/core` | 36 pass | Crate | 5D trajectory, HNSW, IRCP |
| RAG++ Python | ✅ Production | `core/cc-rag-plus-plus/rag_plusplus` | - | Cloud Run | CognitiveTwin, FastAPI |
| Graph Kernel | ✅ Production | `core/cc-graph-kernel` | 140 pass | Cloud Run | Context slicing, HMAC tokens |
| Orbit | ✅ Production | `apps/trajectory/trajectory-orbit` | - | Cloud Run | Session orchestration |
| Agent SDK | ✅ Production | `core/cc-agent-sdk` | - | npm | Claude agent wrapper, MCP tools |
| mocopi-simulator | ✅ Production | `core/cc-relay/src/bin/mocopi_simulator.rs` | 6 pass | CLI | Motion pattern generator |
Deployment Endpoints
| Service | Endpoint | Protocol | Purpose |
|---|---|---|---|
| DO Relay | `[ip]:12351` | UDP | Mocopi packet ingestion |
| DO Metrics | `[ip]:9090` | HTTP | Prometheus metrics |
| GCP VM | `[ip]:8765` | HTTP | Headless daemon API |
| GCP VM WS | `[ip]:8766` | WebSocket | Real-time visualization |
| Orbit | `orbit-server-*.run.app` | HTTPS | Session management |
| RAG++ | `rag-plusplus-*.run.app` | HTTPS | Memory service |
| Graph Kernel | `graph-kernel-*.run.app` | HTTPS | Admissibility authority |
---
Executive Summary
Comp-Core is a production-grade motion-to-meaning pipeline that transforms embodied human movement into musical expression. The system processes real-time sensor data from:
- Sony Mocopi: 6-12 IMU sensors (Base Kit / Pro Kit) → 27-bone skeleton
- iPhone/Apple Watch: CoreMotion IMU data via WebSocket
- MediaPipe: Camera-based pose estimation (33 landmarks)
The pipeline converts raw motion into anticipatory signals (commitment, uncertainty, transition pressure), gesture classifications, and ultimately sound control signals.
Latency Budget (End-to-End: < 10ms)
Mocopi UDP → TLV Parse (50μs) → EKF Fusion (100μs) → Anticipation (2ms) → Gesture (500μs) → Audio (1ms)---
Conceptual Hierarchy
The architecture is organized around three conceptual layers that represent different questions about motion:
┌─────────────────────────────────────────────────────────────┐
│ │
│ COMPUTATIONAL CHOREOGRAPHY │
│ The Discipline │
│ │
│ "What does movement mean, and how can we compute it?" │
│ │
├─────────────────────────────────────────────────────────────┤
│ │
│ TrajectoryOS │
│ The Operating System │
│ │
│ "What does this moment mean in the context of │
│ everything else, and where does it sit in the │
│ long arc?" │
│ │
│ Time Scale: Hours → Years │
│ Operates on: Sessions, projects, creative arcs │
│ Output: Context, memory, style │
│ │
├─────────────────────────────────────────────────────────────┤
│ │
│ Echelon │
│ The Real-Time Engine │
│ │
│ "What is happening now, in the body, and how do we │
│ keep this moment coherent?" │
│ │
│ Time Scale: Milliseconds → Seconds │
│ Operates on: Continuous motion dynamics │
│ Output: Control signals, gestures, audio │
│ │
└─────────────────────────────────────────────────────────────┘Document Links
| Layer | Document | Description |
|---|---|---|
| Philosophy | [01-COMPUTATIONAL_CHOREOGRAPHY.md](01-COMPUTATIONAL_CHOREOGRAPHY.md) | Core principles and vocabulary |
| TrajectoryOS | [02-TRAJECTORY_OS.md](02-TRAJECTORY_OS.md) | Long-horizon operating system |
| Echelon | [03-ECHELON.md](03-ECHELON.md) | Real-time engine architecture |
---
Architecture Diagram
flowchart TB
subgraph Philosophy [Computational Choreography]
CC[Motion as Meaning<br/>The Discipline]
end
subgraph TrajectoryOS [TrajectoryOS - Long Horizon OS]
direction TB
subgraph Memory [Memory Layer]
RAG[RAG++ Memory Fabric<br/>5D Trajectory Coordinates]
CT[CognitiveTwin<br/>Style Signature]
HNSW[HNSW Index<br/>O(log n) ANN]
end
subgraph Orchestration [Orchestration Layer]
Orbit[Orbit Server<br/>Axum/Rust]
Sessions[Session Manager<br/>Supabase]
end
subgraph Trajectory [Trajectory Layer]
TC[5D Coordinates<br/>temporal, semantic, depth, homogeneity, salience]
IRCP[I-RCP Propagator<br/>Attention Weights]
end
end
subgraph Echelon [Echelon - Real-Time Engine]
direction TB
subgraph Sensor [Layer 0-1: Sensor Input]
Mocopi[Mocopi 27-Bone<br/>UDP 12351]
Relay[cc-relay<br/>Rust Binary]
Collection[cc-collection<br/>Extended Kalman Filter]
end
subgraph Semantic [Layer 2-3: Semantic]
Aligner[cc-window-aligner<br/>50 Hz Windows]
Anticipation[cc-anticipation<br/>Commitment, Uncertainty]
Gesture[cc-gesture<br/>Neighbor Voting]
end
subgraph DELL [Layer 3.5: DELL]
DELLCoord[DELLCoordinator]
Fast[Fast Equilibrium<br/>60Hz]
Slow[Slow Equilibrium<br/>~2.5Hz]
end
subgraph Audio [Layer 4-7: Audio]
Brain[cc-brain<br/>Latent Updater]
Conductor[cc-conductor<br/>Beat Scheduler]
DSP[Real-Time DSP<br/>Lock-Free]
end
end
CC --> TrajectoryOS
CC --> Echelon
Mocopi --> Relay --> Collection --> Aligner --> Anticipation
Anticipation --> Gesture --> Brain
Anticipation --> DELLCoord --> Fast --> Slow
DELLCoord --> Brain --> Conductor --> DSP
Echelon -->|Trajectory Segments| TrajectoryOS
TrajectoryOS -->|Context Window| Echelon---
Data Flow Pipeline
Complete Pipeline (Sensor to Audio)
flowchart LR
subgraph Input [Sensor Input Layer]
M[Mocopi UDP<br/>12351<br/>30-60 Hz]
I[iPhone/Watch<br/>WebSocket<br/>60 Hz]
C[MediaPipe<br/>WebSocket<br/>30 Hz]
end
subgraph Relay [cc-relay]
TLV[TLV Parser<br/>Zero-Copy]
Batch[Frame Batching<br/>30 frames]
Sync[Cloud Sync<br/>Circuit Breaker]
end
subgraph Fusion [cc-collection]
EKF[Extended Kalman Filter<br/>13-dim state per limb]
Occ[Occlusion Handler<br/>Exponential Decay]
end
subgraph Window [cc-window-aligner]
WA[Temporal Alignment<br/>Clock Skew Correction]
MW[MotionWindow<br/>50 Hz Canonical]
end
subgraph Anticipation [cc-anticipation]
Kernel[Anticipation Kernel<br/>7 Scalars + 3 Vectors]
Phrase[Phrase Library<br/>HNSW Neighbors]
end
subgraph Gesture [cc-gesture]
Class[Gesture Classifier<br/>Neighbor Voting]
Hand[Hand Kernel<br/>iPhone/Watch]
end
subgraph Brain [cc-brain]
Latent[Latent Updater<br/>State Machine]
Lex[Lexicon<br/>Expressive Controls]
end
subgraph Output [Output Layer]
Audio[Audio Engine<br/>48kHz, < 1ms]
Viz[Visualization<br/>WebSocket]
DJ[DJ Control<br/>MIDI/OSC]
end
M --> TLV --> Batch --> Sync --> EKF
I --> EKF
C --> EKF
EKF --> Occ --> WA --> MW --> Kernel
Kernel --> Phrase --> Class
Hand --> Class
Class --> Latent --> Lex
Lex --> Audio
Lex --> Viz
Class --> DJ---
Crate Dependency Graph
flowchart BT
subgraph Protocol [Protocol Layer]
Protocol[cc-protocol<br/>Types, Schemas, Serialization]
end
subgraph Input [Input Layer]
Relay[cc-relay<br/>TLV Parser, UDP]
Collection[cc-collection<br/>Kalman Fusion]
end
subgraph Semantic [Semantic Layer]
WindowAligner[cc-window-aligner<br/>Temporal Alignment]
Anticipation[cc-anticipation<br/>Commitment, Uncertainty]
Gesture[cc-gesture<br/>Classification]
end
subgraph Engine [Engine Layer]
Brain[cc-brain<br/>Latent Equilibrium]
DELL[dell<br/>Dual Timescale]
Conductor[cc-conductor<br/>Beat Scheduling]
AudioEngine[audio-engine<br/>DSP]
end
subgraph Memory [Memory Layer]
RAGCore[rag_plusplus_core<br/>HNSW, IRCP, 5D]
end
Protocol --> Relay
Protocol --> Collection
Collection --> WindowAligner
WindowAligner --> Anticipation
Anticipation --> Gesture
RAGCore --> Anticipation
Gesture --> Brain
Anticipation --> DELL
DELL --> Brain
Brain --> Conductor --> AudioEngine---
Key Invariants
These invariants are NEVER violated. Violation indicates a bug.
| ID | Invariant | Enforcement | Checked At |
|---|---|---|---|
| INV-001 | Deterministic replay | No random seeds, same input → same output | All kernels |
| INV-002 | Quaternion wxyz order | All quaternions use scalar-first convention | Parsing, fusion |
| INV-003 | Coverage threshold | Windows with coverage < 0.9 are rejected | Kernel entry |
| INV-004 | Scalar bounds | commitment, uncertainty ∈ [0, 1] | Packet creation |
| INV-005 | Bone count | Mocopi skeleton = 27 bones (0-26) | Frame creation |
| INV-006 | No hot-path allocation | Pre-allocated buffers in kernel | Runtime |
| INV-007 | Schema versioning | AnticipationPacket.schema_version = "0.1.0" | Deserialization |
| INV-GK-001 | Slice boundary | `turn_id ∈ slice.turn_ids` enforced | Slice retrieval |
| INV-GK-002 | Provenance completeness | All provenance fields required | Slice export |
| INV-GK-003 | No phantom authority | Missing [sensitive field redacted] |
See: [13-DATA_CONTRACTS.md](13-DATA_CONTRACTS.md), [15-GRAPH_KERNEL.md](15-GRAPH_KERNEL.md) for frozen schemas and admissibility.
---
Performance Targets
| Component | Target Latency | Measured | Notes |
|---|---|---|---|
| TLV Parsing | < 50μs | ~30μs | Zero-copy parsing |
| Kalman Update | < 100μs | ~50μs | Per-limb filter |
| Anticipation Kernel | < 2ms | ~1.5ms | Full window processing |
| DELL Fast Equilibrium | < 500μs | ~300μs | Per-frame |
| Gesture Classification | < 500μs | ~200μs | With neighbor lookup |
| Audio Scheduling | < 1ms | ~500μs | Beat-quantized |
| End-to-end | < 10ms | ~8ms | Sensor to audio |
See: [14-TEST_SUITE.md](14-TEST_SUITE.md) for benchmark commands.
---
Documentation Index
| Document | Description | Key Topics |
|---|---|---|
| [00-OVERVIEW.md](00-OVERVIEW.md) | This document | Architecture summary |
| [01-COMPUTATIONAL_CHOREOGRAPHY.md](01-COMPUTATIONAL_CHOREOGRAPHY.md) | Philosophy and discipline | Anticipation model, vocabulary |
| [02-TRAJECTORY_OS.md](02-TRAJECTORY_OS.md) | Long-horizon OS | RAG++, Orbit, CognitiveTwin |
| [03-ECHELON.md](03-ECHELON.md) | Real-time engine | DELL, cc-brain, audio |
| [04-SENSOR_INPUT.md](04-SENSOR_INPUT.md) | Mocopi, IMU, MediaPipe | TLV protocol, bone IDs |
| [05-SENSOR_FUSION.md](05-SENSOR_FUSION.md) | Extended Kalman Filter | Fusion algorithm, weights |
| [06-ANTICIPATION.md](06-ANTICIPATION.md) | Anticipation kernel | Scalars, vectors, neighbors |
| [07-GESTURE_RECOGNITION.md](07-GESTURE_RECOGNITION.md) | Gesture classification | Neighbor voting, gating |
| [08-RAG_PLUS_PLUS.md](08-RAG_PLUS_PLUS.md) | Memory and retrieval | 5D coordinates, HNSW |
| [09-ORBIT.md](09-ORBIT.md) | Orchestration layer | Project/session management |
| [10-AUDIO_ENGINE.md](10-AUDIO_ENGINE.md) | Beat scheduling and DSP | Conductor, DJ integration |
| [11-VISUALIZATION.md](11-VISUALIZATION.md) | 3D topology rendering | Three.js, WebSocket |
| [12-DEPLOYMENT.md](12-DEPLOYMENT.md) | Infrastructure | GCP, Cloud Run, VPS relay |
| [13-DATA_CONTRACTS.md](13-DATA_CONTRACTS.md) | Frozen schemas | Versioning policy |
| [14-TEST_SUITE.md](14-TEST_SUITE.md) | Testing guide | E2E, benchmarks, CI |
| [15-GRAPH_KERNEL.md](15-GRAPH_KERNEL.md) | Graph Kernel | Admissibility, context slicing |
| [16-GRAPH_KERNEL_INTEGRATION.md](16-GRAPH_KERNEL_INTEGRATION.md) | Integration guide | Client wrappers, RAG++ integration |
| [17-AGENT_SDK.md](17-AGENT_SDK.md) | Agent SDK | Claude wrapper, MCP tools, hooks |
---
Quick Start
1. Test Without Hardware (Mocopi Simulator)
# Build and run the simulator
cd core/cc-relay
cargo run --release --bin mocopi-simulator -- \
--target [ip]:12351 \
--motion dance \
--duration 30 \
--fps 50
# Available motion patterns: idle, walk, dance, wave, expressive2. Verify Data Flow
# Check relay metrics (Prometheus)
curl -s http://[ip]:9090/metrics | grep cc_relay
# Check GCP VM health
curl http://[ip]:8765/health
# Check frame count
curl http://[ip]:8765/mocopi-state | jq '.count'
# Get latest moment
curl http://[ip]:8765/moment | jq3. Run Locally
# Terminal 1: Rust relay
cd core/cc-relay && cargo run --release
# Terminal 2: Headless daemon
cd backend/cc-mcs/src-tauri && cargo run --bin cc-mcs-headless
# Terminal 3: RAG++ service
cd core/cc-rag-plus-plus && python -m rag_plusplus.service.app
# Terminal 4: Orbit server
cd apps/trajectory/trajectory-orbit && cargo run4. Run Tests
# Full test suite
cargo test --workspace --all-features
# Run cc-echelon tests specifically
cd core/cc-echelon && cargo test --workspace
# Run E2E local test
./scripts/test_e2e_local.sh quick
# Run benchmarks
cargo bench --workspace---
Audit Status
Last audit: 2025-01-01
| Priority | Finding | Location | Status |
|---|---|---|---|
| High | No HTTPS on VM | Deployment | ⚠️ Add Caddy |
| High | Session cleanup | Orbit | ⚠️ Add TTL job |
| Medium | Hand pipeline immature | cc-gesture | Planned |
| Medium | Salience weights untuned | RAG++ | Needs data |
| Low | 70/30 blend hardcoded | Anticipation | Make configurable |
| Low | Mobile viz performance | Frontend | Reduce complexity |
See: [13-DATA_CONTRACTS.md](13-DATA_CONTRACTS.md) for consolidated audit findings.
---
Contributing
1. Read the relevant architecture document before making changes
2. Ensure invariants are maintained (especially INV-001, INV-002, INV-005)
3. Run the full test suite: `cargo test --workspace --all-features`
4. Update documentation if public APIs change
5. Add tests for new functionality
Code Style
- Rust: `cargo fmt` and `cargo clippy`
- Python: `black` and `ruff`
- TypeScript: `prettier` and `eslint`
---
Version History
| Version | Date | Changes |
|---|---|---|
| 2.4.0 | 2026-01-03 | Added Agent SDK documentation, updated component matrix |
| 2.3.0 | 2026-01-02 | Added Graph Kernel production deployment, admissibility tokens |
| 2.2.0 | 2025-01-01 | Added DELL, comprehensive test suite, Rust relay |
| 2.1.0 | 2024-12-15 | Added mocopi-simulator, updated deployment |
| 2.0.0 | 2024-12-01 | Echelon/TrajectoryOS split, RAG++ integration |
| 1.0.0 | 2024-11-01 | Initial architecture documentation |
Promotion Decision
Promote into a technical note or architecture paper with implementation anchors.
Source Anchor
Comp-Core/docs/architecture/00-OVERVIEW.md
Detected Structure
Method · Evaluation · Code Anchors · Architecture