Grand Diomande Research · Full HTML Reader

Echelon

**Version**: 1.2.0 **Last Updated**: 2025-01-01 **Status**: Production **Parent**: [00-OVERVIEW.md](00-OVERVIEW.md) **Philosophy**: [01-COMPUTATIONAL_CHOREOGRAPHY.md](01-COMPUTATIONAL_CHOREOGRAPHY.md) **Complement**: [02-TRAJECTORY_OS.md](02-TRAJECTORY_OS.md) (long-horizon OS)

Embodied Trajectory Systems architecture technical paper candidate score 48 .md

Full Public Reader

Echelon

The Real-Time Embodied Engine

Version: 1.2.0
Last Updated: 2025-01-01
Status: Production
Parent: [00-OVERVIEW.md](00-OVERVIEW.md)
Philosophy: [01-COMPUTATIONAL_CHOREOGRAPHY.md](01-COMPUTATIONAL_CHOREOGRAPHY.md)
Complement: [02-TRAJECTORY_OS.md](02-TRAJECTORY_OS.md) (long-horizon OS)

---

Overview

Echelon is the real-time embodied physics engine that processes motion at millisecond timescales. It answers:

> What is happening now, in the body, and how do we keep this moment coherent?

Echelon owns the physics of embodiment: latency, stability, coherence, and real-time response. Nothing in TrajectoryOS or RAG++ can replace this capability.

Architectural Law

This distinction is architectural law, not a branding question:

Regime	Echelon	TrajectoryOS
Latency	Milliseconds (existential)	Seconds (acceptable)
State	Continuous dynamical manifold	Discrete memory turns
Question	How do we keep coherence?	What does this mean?
Mode	Act, not reason	Reason, then act

---

Key Characteristics

Time Constants

Layer	Latency Budget	Frame Rate
Sensor Input	< 20ms	30-60 Hz
Fusion	< 5ms	60 Hz
Anticipation	< 2ms	50 Hz
Audio Scheduling	< 1ms	48kHz

Operating Regime

Echelon operates in a regime where:
- Milliseconds matter
- Latency is existential
- Instability must be corrected before it propagates
- The system must act, not just reason

---

Architecture

mermaid

flowchart TB
    subgraph Sensor [Layer 0: Sensor Input]
        Mocopi[Mocopi UDP]
        IMU[iPhone/Watch IMU]
        Camera[MediaPipe]
        Relay[cc-relay]
    end

    subgraph Fusion [Layer 1: Sensor Fusion]
        Collection[cc-collection]
        Kalman[Extended Kalman Filter]
        Occlusion[Occlusion Handler]
    end

    subgraph Window [Layer 2: Windowing]
        Aligner[cc-window-aligner]
        MotionWindow[MotionWindow]
    end

    subgraph Semantic [Layer 3: Semantic Extraction]
        Anticipation[cc-anticipation]
        Gesture[cc-gesture]
    end

    subgraph DELL [Layer 3.5: DELL]
        DELLCoord[DELLCoordinator]
        FastEq[Fast Equilibrium<br/>60Hz]
        SlowEq[Slow Equilibrium<br/>~2.5Hz]
    end

    subgraph Brain [Layer 4: Latent Equilibrium]
        CCBrain[cc-brain]
        Latent[Latent Updater]
        StateMachine[State Machine]
        Lexicon[Lexicon Controller]
    end

    subgraph Audio [Layer 5-7: Audio]
        Conductor[cc-conductor]
        Scheduler[scheduler]
        AudioEngine[audio-engine]
    end

    Mocopi --> Relay --> Collection
    IMU --> Collection
    Camera --> Collection
    Collection --> Kalman --> Occlusion --> Aligner
    Aligner --> MotionWindow --> Anticipation
    Anticipation --> Gesture
    Anticipation --> DELLCoord
    DELLCoord --> FastEq --> SlowEq
    Gesture --> CCBrain
    DELLCoord --> CCBrain
    CCBrain --> Latent --> StateMachine --> Lexicon
    Lexicon --> Conductor --> Scheduler --> AudioEngine

---

Layer 0: Sensor Input

Mocopi Protocol

Sony Mocopi sends UDP packets with TLV (Type-Length-Value) structure:

┌─────────┬─────────┬─────────────────────────────┐
│ Length  │   Tag   │           Value             │
│ 4 bytes │ 4 bytes │        Length bytes         │
└─────────┴─────────┴─────────────────────────────┘

Tags:
- `skdf`: Skeleton data frame
- `fram`: Frame data
- `btrs`: Bone transforms
- `btdt`: Bone data
- `bnid`: Bone ID
- `tran`: Transform (7 floats: qx, qy, qz, qw, px, py, pz)

See: [04-SENSOR_INPUT.md](04-SENSOR_INPUT.md)

cc-relay

The Rust-based relay provides:
- Zero-copy TLV parsing
- Quaternion conversion (xyzw → wxyz)
- Cloud sync with circuit breaker
- Prometheus metrics

Location: `core/gateways/cc-relay/`

---

Layer 1: Sensor Fusion

Extended Kalman Filter

Per-limb EKF with 13-dimensional state:

Index	Component	Description
0-2	Position	World [x, y, z] meters
3-6	Quaternion	[w, x, y, z] orientation
7-9	Linear Velocity	[vx, vy, vz] m/s
10-12	Angular Velocity	[wx, wy, wz] rad/s

Sensor Weights:

Sensor	Orientation Weight	Position Weight
Mocopi	0.95	0.70
MediaPipe	0.40	0.80

See: [05-SENSOR_FUSION.md](05-SENSOR_FUSION.md)

---

Layer 2: Windowing

cc-window-aligner

Produces deterministic `MotionWindow` structs at canonical FPS (50 Hz):

rust

pub struct MotionWindow {
    pub window_id: String,      // Deterministic hash
    pub t_start: f64,
    pub t_end: f64,
    pub fps: f32,
    pub skeleton_frames: Vec<SkeletonFrame>,
    pub latent_frames: Vec<LatentFrame>,
    pub coverage: f32,          // [0, 1]
    pub device_offsets: HashMap<String, f64>,
}

---

Layer 3: Semantic Extraction

cc-anticipation

Converts MotionWindow to AnticipationPacket:

rust

pub struct AnticipationPacket {
    // Scalars (all [0, 1] except transition_pressure)
    pub commitment: f32,
    pub uncertainty: f32,
    pub transition_pressure: f32,
    pub recovery_margin: f32,
    pub phase_stiffness: f32,
    pub novelty: f32,
    pub stability: f32,

    // Vectors
    pub regime_embedding: Vec<f32>,    // 64-256 dims
    pub constraint_vector: Vec<f32>,   // ~8 dims
    pub derivative_summary: Vec<f32>,  // ~8 dims

    // Provenance
    pub window_id: String,
    pub timestamp: f64,
    pub schema_version: String,
}

See: [06-ANTICIPATION.md](06-ANTICIPATION.md)

cc-gesture

Gesture classification using neighbor voting:

rust

pub struct GestureClassifier {
    config: GestureConfig,
    labels: HashMap<u32, GestureLabel>,
    phrase_labels: HashMap<String, Vec<u32>>,
    metrics: ClassifierMetrics,
}

Algorithm:
1. Query neighbors from phrase library
2. Aggregate votes with exponential rank decay
3. Compute confidence from votes + anticipation
4. Apply commitment gating
5. Apply cooldown

See: [07-GESTURE_RECOGNITION.md](07-GESTURE_RECOGNITION.md)

---

Layer 3.5: DELL (Dual-Equilibrium Latent Learning)

Location: `core/cc-echelon/crates/dell`

DELL provides beat-aligned motion processing with two timescales:

Architecture

rust

pub struct DELLCoordinator {
    fast_equilibrium: FastEquilibrium,  // 60Hz (per-frame)
    slow_equilibrium: SlowEquilibrium,  // ~2.5Hz (beat-quantized)
    coupler: Coupler,                   // Bidirectional coupling
}

Fast Equilibrium (60Hz)

Captures micro-dynamics of the body:

rust

pub struct FastEquilibrium {
    pub w_in: DMatrix<f32>,        // Input projection
    pub w_recurrent: DMatrix<f32>, // Recurrent weights
    pub w_out: DMatrix<f32>,       // Output projection
    pub h: DVector<f32>,           // Hidden state
}

Fixed-point iteration:

for _ in 0..K_fast:
    h = tanh(W_in @ limbs + W_recurrent @ h + bias)
x_fast = W_out @ h

Slow Equilibrium (~2.5Hz)

Maintains musical/temporal coherence:

rust

pub struct SlowEquilibrium {
    pub pooling: PoolType,  // Mean/Max/Attention
    pub k_slow: usize,      // Iterations (default: 6)
    pub d_slow: usize,      // Latent dimension (default: 32)
}

DELLState Output

rust

pub struct DELLState {
    pub x_fast: Vec<f32>,           // Fast latent state
    pub y_slow: Vec<f32>,           // Slow latent state
    pub beat_phase: f32,            // [0, 1) within current beat
    pub tempo_bpm: f32,             // Current tempo
    pub frame_idx: u64,
    pub fast_convergence: f32,      // Fixed-point residual
}

Integration Point

rust

// In cc-mcs-headless main loop
let dell_state = dell_bridge.process_anticipation(&packet).await;
sensors::broadcast_dell_state(&dell_state);

---

Layer 4: Latent Equilibrium

cc-brain

The EchelonCore processes sensor frames into expressive outputs:

rust

pub struct EchelonCore {
    latent_updater: Box<dyn LatentUpdater>,
    state_machine: SectionStateMachine,
    lexicon_controller: LexiconController,
}

Latent State

rust

pub struct LatentState {
    pub z: Vec<f32>,          // Latent vector
    pub velocity: Vec<f32>,   // dz/dt
    pub norm: f32,
    pub speed: f32,
    pub curvature: f32,
    pub periodicity: f32,
}

Section State Machine

Musical section states:

rust

pub enum SectionState {
    Entry,      // Beginning
    Build,      // Increasing intensity
    Peak,       // Maximum intensity
    Release,    // Decreasing intensity
    Ambient,    // Low intensity steady state
}

Lexicon

Expressive control signals:

rust

pub struct Lexicon {
    pub tension: f32,
    pub energy: f32,
    pub divergence: f32,
    pub expressivity: f32,
    pub transition_intensity: f32,
    pub resolution: f32,
}

---

Layer 5-7: Audio

cc-conductor

Beat-quantized control of DJ actions:

rust

pub struct Conductor {
    pub tempo_bpm: f32,
    pub time_signature: TimeSignature,
    pub current_beat: f32,
}

Scheduler

Quantizes actions to beat grid:

rust

pub struct BeatScheduler {
    pub tempo: Tempo,
    pub quantize: QuantizeMode,
    pub pending_actions: Vec<ScheduledAction>,
}

Audio Engine

Real-time DSP with hard latency guarantees:

Budget	Usage
< 1ms	Audio callback
< 100μs	Per-sample processing
0 allocations	In audio thread

---

Safety Properties

Real-Time Guarantees

1. No allocation in audio thread: Pre-allocated buffers
2. No blocking: Lock-free communication
3. Bounded latency: Fixed buffer sizes
4. Graceful degradation: Skip frames rather than accumulate latency

Numerical Stability

1. Quaternion normalization: After every update
2. Clamp extreme values: Position, velocity, angular velocity
3. NaN/Inf detection: With fallback to identity/zero
4. Deterministic: No random seeds, same input → same output

---

Metrics and Observability

Prometheus Metrics

echelon_frames_processed_total
echelon_anticipation_latency_seconds
echelon_gesture_detections_total
echelon_audio_underruns_total

Tracing

rust

#[instrument(name = "process_frame", fields(frame_id = %frame.id))]
fn process(&mut self, frame: &SensorFrame) -> Result<()> {
    // ...
}

---

Configuration

AnticipationConfig

rust

pub struct AnticipationConfig {
    pub min_coverage: f32,           // 0.9
    pub regime_embedding_dim: usize, // 64
    pub transition_pressure_alpha: f32, // 0.3
    pub emit_debug: bool,
}

GestureConfig

rust

pub struct GestureConfig {
    pub min_confidence: f32,      // 0.6
    pub commitment_threshold: f32, // 0.7
    pub completion_threshold: f32, // 0.5
    pub cooldown_micros: u64,     // 500_000
    pub distance_decay: f32,      // 0.2
}

---

Promotion Decision

Promote into a technical note or architecture paper with implementation anchors.

Source Anchor

Comp-Core/docs/architecture/03-ECHELON.md

Detected Structure

Method · Evaluation · Architecture

Full Public Reader

Echelon

Overview

Architectural Law

Key Characteristics

Time Constants

Operating Regime

Architecture

Layer 0: Sensor Input

Mocopi Protocol

cc-relay

Layer 1: Sensor Fusion

Extended Kalman Filter

Layer 2: Windowing

cc-window-aligner

Layer 3: Semantic Extraction

cc-anticipation

cc-gesture

Layer 3.5: DELL (Dual-Equilibrium Latent Learning)

Architecture

Fast Equilibrium (60Hz)

Slow Equilibrium (~2.5Hz)

DELLState Output

Integration Point

Layer 4: Latent Equilibrium

cc-brain

Latent State

Section State Machine

Lexicon

Layer 5-7: Audio

cc-conductor

Scheduler

Audio Engine

Safety Properties

Real-Time Guarantees

Numerical Stability

Metrics and Observability

Prometheus Metrics

Tracing

Configuration

AnticipationConfig

GestureConfig

Further Reading

Advanced Deep-Dive Documentation ✨

Promotion Decision

Source Anchor

Detected Structure