EchelonCapture: Current State vs. Vision

EchelonCapture: Current State vs. Vision

Executive Summary

EchelonCapture is currently a sensor data streaming & recording app.
It needs to become the visual performance dashboard for Computational Choreography.

---

🎯 What EchelonCapture SHOULD Be

Based on your performance loop diagram and UI vision documents, EchelonCapture should be the real-time visual manifestation of the entire CC system - showing the dancer what the machine sees, what it's generating, and where the performance is going.

The Vision (from your docs):

EchelonCapture = The Performance Mirror

It should display:
1. Latent Orb - Living visualization of LIM-RPS equilibrium (the machine's perception of you)
2. Phrase Spine - Current generative phrase as topological object
3. Generative Horizon - Predicted future latent states (what's coming)
4. Phrase Reservoir - Cloud of upcoming musical possibilities
5. Embodied Control Band - Real-time modulation parameters (tension, grounding, etc.)
6. Somatic Timeline - Compressed history of latent evolution
7. Audio Vessel - Musical form visualization
8. Ambient Aura - Global performance state atmosphere

---

📱 What EchelonCapture IS Currently

A 5-tab iOS app for sensor streaming:

### Current Tabs:
1. Stream - Connect to cc-mcs-headless, stream sensor data, view FPS/latency
2. Record - Save sensor sessions locally
3. Sensors - Configure which sensors are active
4. Sessions - Browse saved recordings
5. Settings - Backend URL, device role, debug mode

### Current Visualization:
- ❌ No latent visualization
- ❌ No music/phrase visualization
- ❌ No generative prediction display
- ✅ Raw sensor debug view (accel, gyro, attitude)
- ✅ Motion energy meter (simple)
- ✅ Network stats (FPS, latency)

---

🔄 The Performance Loop & EchelonCapture's Role

┌─────────────────────────────────────────────────────────────┐
│  📱 SENSORS (iPhones, Watch, AirPods)                        │
│      ↓                                                        │
│  ┌──────────────────────────────────────┐                    │
│  │ cc-mcs-headless (RUST)               │                    │
│  │ • Sensor fusion                      │                    │
│  │ • LatentState computation            │ ← EchelonCapture   │
│  │ • Section state machine              │   streams HERE     │
│  └──────────────────────────────────────┘                    │
│      ↓ Current LatentState                                   │
│  ┌──────────────────────────────────────┐                    │
│  │ COMPUTATIONAL REHEARSAL (RUST)       │                    │
│  │ "What if you keep moving like this?" │ ← Should RECEIVE   │
│  │ → LatentTrajectory (46 frames)       │   this back and    │
│  └──────────────────────────────────────┘   VISUALIZE it     │
│      ↓                                                        │
│  ┌──────────────────────────────────────┐                    │
│  │ PATTERN CODER (PYTHON)               │ ← Should RECEIVE   │
│  │ → [PatternEdit, PatternEdit, ...]   │   these and        │
│  └──────────────────────────────────────┘   VISUALIZE them   │
│      ↓                                                        │
│  ┌──────────────────────────────────────┐                    │
│  │ MUSIC ENGINES                        │ ← Should VISUALIZE │
│  │ • Strudel/Tone.js                    │   what's playing   │
│  │ • Neural Audio (EchelonDiT)          │                    │
│  └──────────────────────────────────────┘                    │
└─────────────────────────────────────────────────────────────┘

Right now: EchelonCapture sends sensor data → cc-mcs-headless (one-way)
Should be: EchelonCapture ⇄ full bidirectional visualization of entire loop

---

🔨 What Needs to Be Added

1. WebSocket Connection to cc-mcs (Backend → App)

Currently EchelonCapture only sends data. It needs to receive:

// NEW: Receive from cc-mcs
struct LatentStateUpdate {
    let position: SIMD3<Float>  // 3D latent position
    let velocity: SIMD3<Float>  // latent velocity
    let equilibrium: Float      // LIM-RPS equilibrium metric
    let tension: Float          // micro-tension
    let grounding: Float        // grounding metric
    let section: String         // current section name
}

struct TrajectoryPrediction {
    let predictedPositions: [SIMD3<Float>]  // 46 frames ahead
    let confidence: Float
}

struct PatternUpdate {
    let activePattern: String
    let patternPhase: Float  // 0.0 - 1.0 through pattern
    let nextPatternCandidate: String?
}

struct AudioState {
    let phraseId: String
    let phraseEnergy: Float
    let phraseProgress: Float  // 0.0 - 1.0
    let harmonic Density: Float
}

2. New Tab: "Performance" (Main View)

This becomes the primary view during a performance. Replace "Stream" tab focus.

Layout (portrait iPhone):

┌─────────────────────────┐
│    🌀 Latent Orb        │  ← Center, largest element
│   (3D visualization)     │     Pulses, stretches, glows
│                          │     based on latent state
│   ╱╲ force lines ╱╲     │
├──────────────────────────┤
│                          │
│  ───🎵 Phrase Spine───  │  ← Flowing ribbon
│  (current phrase)        │     Color = harmonic density
│                          │     Thickness = energy
├──────────────────────────┤
│ 📊 Embodied Params       │  ← Waveform-like shapes
│ ▂▃▅▂ Tension            │     Showing extracted
│ ▃▅▃▂ Grounding          │     latent features
│ ▂▂▅▃ Verticality        │
├──────────────────────────┤
│ ═══ Somatic Timeline    │  ← Trailing history
│     (last 3 sec)        │     Glows where tension
└──────────────────────────┘     was high

3. Metal/SceneKit for 3D Latent Orb

The orb needs to be a real-time 3D object:

class LatentOrbRenderer {
    // 3D sphere that deforms based on latent state
    // - Stretches in direction of velocity
    // - Ripples when micro-tension spikes
    // - Glows with intensity = energy
    // - Color shift based on section state
}

4. Phrase Spine as Animated Path

struct PhraseSpineView: View {
    let phraseState: AudioState

    // Bezier curve that:
    // - Flows left→right
    // - Thickness varies with energy
    // - Hue varies with harmonic density
    // - Animates as phrase progresses
}

5. Generative Horizon (Future Prediction)

struct GenerativeHorizonView: View {
    let trajectory: TrajectoryPrediction

    // Translucent corridor showing
    // where latent *might* go in next 2 sec
    // Bends forward when dancer accelerates
    // Flattens when dancer pauses
}

---

🎨 Visual Design Language

### Colors (Ambient Aura)
- Blue-Purple: Low energy, exploratory
- Purple-Magenta: Medium energy, building
- Magenta-Red: High energy, intense
- Gold-Orange: Transitional states
- Green-Cyan: Grounded, stable sections

### Animation Timing
- Latent updates: 50 FPS (matches sensor rate)
- Phrase spine: Smooth 60 FPS Metal rendering
- Horizon updates: 20 FPS (predicted states)
- Aura transitions: 2-3 second eases

### Typography
- Section names: SF Pro Display Bold, 24pt
- Metrics: SF Mono, 12pt (for debug)
- No timecodes, no BPM - only embodied time

---

🏗️ Architecture Changes Needed

Current:

EchelonCapture → cc-mcs-headless
   (sensors)        (compute)

Needed:

EchelonCapture ⇄ cc-mcs-headless ⇄ cc-ml (Pattern Coder)
   (sensors)      (latent compute)    (musical decisions)
      ↑                 ↓                    ↓
      └─────────── visualization ───────────┘
                   (latent, trajectory,
                    patterns, audio)

New Services Needed:

class LatentVisualizationService: ObservableObject {
    @Published var currentLatent: LatentStateUpdate?
    @Published var trajectory: TrajectoryPrediction?

    // Subscribe to WebSocket updates from cc-mcs
    func connectToVisualizationStream(url: String)
}

class PatternVisualizationService: ObservableObject {
    @Published var activePattern: PatternUpdate?
    @Published var phraseReservoir: [String]  // upcoming candidates

    // Subscribe to Pattern Coder updates
}

class AudioVisualizationService: ObservableObject {
    @Published var currentPhrase: AudioState?
    @Published var audioVessel: VesselMetrics  // form, not amplitude
}

---

📊 Feature Comparison

---

🎯 Recommended Implementation Path

### Phase 1: Backend Communication (Week 1)
1. Add WebSocket receive capability to NetworkService
2. Define data models for LatentState, Trajectory, Pattern, Audio
3. Create visualization services (Latent, Pattern, Audio)
4. Test data flow: cc-mcs → EchelonCapture

### Phase 2: Latent Orb (Week 2)
1. Build 3D SceneKit orb renderer
2. Map latent state → orb deformation
3. Add force field visualization around orb
4. Integrate into new PerformanceView

### Phase 3: Phrase Visualization (Week 3)
1. Design Phrase Spine as animated Bezier path
2. Map audio state → spine properties
3. Add Generative Horizon (trajectory corridor)
4. Test with live performance

### Phase 4: Complete Dashboard (Week 4)
1. Add Embodied Control Band (tension, grounding, etc.)
2. Add Somatic Timeline (3-second history)
3. Add Audio Vessel (form meter)
4. Add Ambient Aura (background color field)
5. Polish animations and timing

### Phase 5: Integration Testing
1. Full system test: sensors → cc-mcs → rehearsal → pattern coder → audio → visualization
2. Performance testing with real dancers
3. Refinement based on actual use

---

💡 Key Design Principles

### 1. The Latent is the Truth
Everything radiates from the latent orb. It is the performer's embodied state made visible.

### 2. No DJ Metaphors
- No decks
- No crossfaders
- No track lists
- No BPM grids

Only: body → latent → music → form

### 3. Embodied Time, Not Clock Time
- No timestamps
- No beat counters
- Only: tension, release, anticipation, resolution

### 4. Generative, Not Manipulative
The UI shows what's being born from your movement, not what you're controlling.

### 5. Unity Over Fragmentation
All visualizations are one organism. The orb, spine, horizon, and aura breathe together.

---

🚀 Next Steps

### Immediate (This Week):
1. ✅ Design WebSocket protocol for cc-mcs → EchelonCapture
2. ✅ Implement LatentVisualizationService
3. ✅ Create basic PerformanceView layout
4. ✅ Build simple latent orb prototype (2D first)

### Short-term (Next 2 Weeks):
1. Full 3D latent orb with SceneKit
2. Phrase spine visualization
3. Live testing with cc-mcs streaming

### Medium-term (Next Month):
1. Complete all visualization components
2. Integration with Pattern Coder
3. Audio visualization
4. Full performance dashboard

---

🎭 The Vision in One Sentence

EchelonCapture should be a window into the machine's mind - showing you what it sees, what it predicts, and what music it's birthing from your movement.

Right now it's a sensor debugger.
It needs to become a living mirror of computational choreography.