Grand Diomande Research · Full HTML Reader

WebSocket Protocol: cc-mcs ↔ EchelonCapture

Currently EchelonCapture **sends** sensor data to cc-mcs via HTTP POST. Now we need **bidirectional communication** so EchelonCapture can **receive** visualization data.

Embodied Trajectory Systems research note experiment writeup candidate score 24 .md

Full Public Reader

WebSocket Protocol: cc-mcs ↔ EchelonCapture

Overview

Currently EchelonCapture sends sensor data to cc-mcs via HTTP POST.
Now we need bidirectional communication so EchelonCapture can receive visualization data.

---

Architecture

EchelonCapture                     cc-mcs-headless
     │                                    │
     │─── POST /sensor_data ────────────→│ (existing)
     │    (sensor frames)                 │
     │                                    │
     │                                    │ Compute:
     │                                    │ • Sensor fusion
     │                                    │ • LIM-RPS latent
     │                                    │ • Trajectory prediction
     │                                    │ • Section state
     │                                    │
     │←── WebSocket /visualization ──────│ (NEW)
     │    (latent, trajectory, patterns)  │
     │                                    │

---

WebSocket Endpoint

Connection

ws://[cc-mcs-url]/visualization?device_id=[left|right|watch]

Query params:
- `device_id`: Which device this is (left, right, watch)
- Optional: `subscribe`: Comma-separated list of channels (latent,trajectory,pattern,audio)

### Authentication
None required for now (same network assumption as existing HTTP)

---

Message Format

All messages are JSON with a `type` field:

json

{
  "type": "latent_state" | "trajectory" | "pattern" | "audio" | "section",
  "timestamp": 1234567890.123,
  "data": { ... }
}

---

Message Types

1. `latent_state` - Real-time Latent Position

Sent at 50 Hz (matches sensor rate)

json

{
  "type": "latent_state",
  "timestamp": 1234567890.123,
  "data": {
    "position": [0.23, -0.45, 0.67],        // 3D latent space position
    "velocity": [0.01, -0.02, 0.03],        // latent velocity vector
    "equilibrium": 0.82,                     // LIM-RPS equilibrium metric (0-1)
    "energy": 1.45,                          // motion energy

    // Extracted embodied features
    "tension": 0.34,                         // micro-tension (0-1)
    "grounding": 0.78,                       // grounding metric (0-1)
    "verticality": 0.65,                     // vertical vs horizontal motion
    "rotation_drive": 0.23,                  // rotational acceleration
    "micro_beat_drift": 0.12                 // rhythmic micro-variation
  }
}

2. `trajectory` - Predicted Future States

Sent at 20 Hz (when trajectory updates)

json

{
  "type": "trajectory",
  "timestamp": 1234567890.123,
  "data": {
    "predicted_positions": [               // 46 frames @ 23fps = ~2 sec
      [0.23, -0.45, 0.67],
      [0.24, -0.44, 0.68],
      // ... 44 more frames
    ],
    "confidence": 0.87,                     // prediction confidence (0-1)
    "curvature": 0.34,                      // trajectory curvature metric
    "anticipated_inflection": true          // whether an inflection point is predicted
  }
}

3. `pattern` - Pattern Coder Decisions

Sent when pattern changes (~1-4 times per second)

json

{
  "type": "pattern",
  "timestamp": 1234567890.123,
  "data": {
    "active_pattern": "buildup_023",        // current active pattern ID
    "pattern_phase": 0.34,                  // progress through pattern (0-1)
    "pattern_energy": 0.67,                 // pattern energy level

    // Next pattern candidates
    "candidates": [
      {
        "pattern_id": "drop_012",
        "probability": 0.45,
        "reason": "high_energy_trajectory"
      },
      {
        "pattern_id": "sustain_008",
        "probability": 0.32,
        "reason": "equilibrium_stable"
      }
    ],

    // Recent pattern edits
    "recent_edits": [
      {
        "edit_type": "modulate_filter",
        "target": "bass_synth",
        "value": 0.78,
        "reason": "tension_increase"
      }
    ]
  }
}

4. `audio` - Audio State & Phrase Info

Sent at ~30 Hz during playback

json

{
  "type": "audio",
  "timestamp": 1234567890.123,
  "data": {
    "phrase_id": "phrase_gen_0234",         // current phrase identifier
    "phrase_type": "generative" | "sample", // generated vs. pre-made
    "phrase_progress": 0.56,                // progress through phrase (0-1)

    // Musical properties
    "harmonic_density": 0.67,               // harmonic complexity (0-1)
    "rhythmic_density": 0.45,               // rhythmic complexity (0-1)
    "spectral_centroid": 2340.5,            // brightness (Hz)

    // Form metrics (NOT amplitude)
    "phrase_energy": 0.78,                  // musical energy
    "tension_curve": 0.34,                  // musical tension (0-1)
    "form_phase": "rising" | "peak" | "falling" | "trough"
  }
}

5. `section` - Section State Machine

Sent when section changes (~every 30-90 seconds)

json

{
  "type": "section",
  "timestamp": 1234567890.123,
  "data": {
    "section_id": "section_3",
    "section_name": "exploration",          // exploration, buildup, peak, release, etc.
    "section_duration": 45.6,               // how long we've been in this section (sec)

    // Transition info
    "in_transition": false,
    "next_section_candidate": "buildup",
    "transition_probability": 0.23
  }
}

6. `ambient_aura` - Global Performance State

Sent at ~5 Hz (slow updates for atmosphere)

json

{
  "type": "ambient_aura",
  "timestamp": 1234567890.123,
  "data": {
    "hue": 280,                             // 0-360 degrees (HSL)
    "saturation": 0.65,                     // 0-1
    "brightness": 0.45,                     // 0-1
    "pulse_rate": 0.5,                      // breathing rate (Hz)

    "global_energy_level": 0.67,            // overall performance energy
    "expressive_mode": "intense" | "exploratory" | "stable" | "transitioning"
  }
}

---

Client-Side Subscription

EchelonCapture can subscribe to specific channels:

swift

// Subscribe to everything
ws://cc-mcs/visualization?device_id=left

// Subscribe only to latent and audio (lighter bandwidth)
ws://cc-mcs/visualization?device_id=left&subscribe=latent,audio

---

Update Frequencies

Message Type	Frequency	Bandwidth	Priority
`latent_state`	50 Hz	~2 KB/s	🔴 Critical
`trajectory`	20 Hz	~3 KB/s	🟡 High
`audio`	30 Hz	~1 KB/s	🟡 High
`pattern`	1-4 Hz	~0.5 KB/s	🟢 Medium
`section`	0.01-0.05 Hz	~0.01 KB/s	🟢 Medium
`ambient_aura`	5 Hz	~0.2 KB/s	⚪ Low

Total bandwidth: ~7 KB/s (very light)

---

Error Handling

Connection Loss

json

{
  "type": "error",
  "timestamp": 1234567890.123,
  "data": {
    "code": "CONNECTION_LOST",
    "message": "WebSocket connection closed",
    "recoverable": true
  }
}

EchelonCapture should:
1. Show connection lost indicator
2. Attempt reconnect every 2 seconds
3. Continue sensor streaming (HTTP still works)
4. Resume visualization when reconnected

### Latency Warning
If round-trip latency > 100ms, cc-mcs can send:

json

{
  "type": "warning",
  "data": {
    "code": "HIGH_LATENCY",
    "latency_ms": 156,
    "recommendation": "reduce_visualization_quality"
  }
}

---

Implementation Priority

### Phase 1 (Week 1):
- ✅ Implement WebSocket server in cc-mcs (Rust/Actix)
- ✅ Send `latent_state` messages only
- ✅ EchelonCapture connects and receives

### Phase 2 (Week 2):
- ✅ Add `trajectory` messages
- ✅ Add `audio` messages
- ✅ Basic visualization with these 3 types

### Phase 3 (Week 3):
- ✅ Add `pattern` messages
- ✅ Add `section` messages
- ✅ Add `ambient_aura` messages
- ✅ Full visualization dashboard

---

Swift Data Models

See `VisualizationModels.swift` for implementation:

swift

struct LatentStateUpdate: Codable {
    let position: SIMD3<Float>
    let velocity: SIMD3<Float>
    let equilibrium: Float
    let energy: Float
    let tension: Float
    let grounding: Float
    // ... etc
}

struct TrajectoryUpdate: Codable {
    let predictedPositions: [SIMD3<Float>]
    let confidence: Float
    let curvature: Float
    let anticipatedInflection: Bool
}

// ... etc

---

Testing

### Mock WebSocket Server
For testing without full cc-mcs:

python

# test_ws_server.py
import asyncio
import websockets
import json
import time

async def mock_visualization(websocket):
    while True:
        # Send mock latent state
        msg = {
            "type": "latent_state",
            "timestamp": time.time(),
            "data": {
                "position": [0.0, 0.0, 0.0],
                "velocity": [0.01, 0.0, 0.0],
                "equilibrium": 0.8,
                "energy": 1.2,
                "tension": 0.3,
                "grounding": 0.7,
                "verticality": 0.5,
                "rotation_drive": 0.1,
                "micro_beat_drift": 0.05
            }
        }
        await websocket.send(json.dumps(msg))
        await asyncio.sleep(0.02)  # 50 Hz

asyncio.run(websockets.serve(mock_visualization, "localhost", 8765))

bash

# Run mock server
python3 test_ws_server.py

# Connect from EchelonCapture
ws://localhost:8765/visualization?device_id=test

---

Next Steps

1. cc-mcs: Add WebSocket endpoint to existing Actix server
2. EchelonCapture: Implement WebSocket client service
3. Test: Mock server → EchelonCapture visualization
4. Integrate: Full cc-mcs → latent computation → WebSocket broadcast

Promotion Decision

Attach run IDs, datasets, metrics, and reproduction commands.

Source Anchor

Comp-Core/apps/ios/EchelonCapture/docs/legacy/WEBSOCKET_PROTOCOL.md

Detected Structure

Method · Evaluation · Code Anchors · Architecture