Grand Diomande Research ยท Full HTML Reader

๐Ÿ“ฑ EchelonCapture Restructuring Plan

**Date**: December 21, 2025 **Status**: ๐Ÿ”„ Planning Phase **Context**: Multi-sensor ecosystem with Mocopi + Strudel music generation

Embodied Trajectory Systems proposal experiment writeup candidate score 24 .md

Full Public Reader

๐Ÿ“ฑ EchelonCapture Restructuring Plan

Date: December 21, 2025
Status: ๐Ÿ”„ Planning Phase
Context: Multi-sensor ecosystem with Mocopi + Strudel music generation

---

๐Ÿ“‹ Executive Summary

With the addition of Sony Mocopi sensors (6 body-mounted IMUs) and the existing Strudel music generation engine, we need to restructure EchelonCapture to fit into a coherent multi-sensor, multi-app ecosystem.

### Current Situation
- 3 iOS Apps: EchelonCapture, cc-handguard, TrajectoryOS
- Sensors: iPhone (2x), Apple Watch, AirPods, Mocopi (6x new)
- Music Generation: Strudel (JavaScript/Web Audio) + optional Neural Audio Server
- Backend: cc-mcs Cloud Run (Rust) for latent physics + Computational Rehearsal

### Key Question from User
> "what's also to be considered from the iOS perspectives how many iPhone applications are to we to be creating as you know we have Strudel and the music generation just wondering if that should be all together"

### Answer
Strudel music generation runs in JavaScript (browser or WKWebView). We have 3 distinct iOS apps with different purposes:

1. EchelonCapture - Motion capture + local Strudel playback
2. cc-handguard - Behavioral intervention (nail-biting detection)
3. TrajectoryOS - Life trajectory optimization

Recommendation: Keep them separate. Each has a distinct use case and user journey.

---

๐ŸŽฏ What is Strudel?

From [cc-rehearsal.md](apps/ios/EchelonCapture/cc-rehearsal.md):

> Strudel is a JavaScript music runtime - a pattern-based music engine inspired by live coding.
>
> - Declarative: music is code (patterns, transforms, functions)
> - Hot-swappable: you can change the pattern graph while sound is playing
> - Time-aware: thinks in cycles, measures, "every 4", "sometimes", etc.
> - Browser-native: runs using Web Audio API (Tone.js)

Where Strudel Runs on iOS

Three integration patterns:

1. Browser-based (Current cc-dashboard): Strudel runs in Safari, audio plays from browser
2. WKWebView embedded (Recommended for EchelonCapture): Native Swift app with embedded web view running Strudel
3. Native Swift reimplementation: Recreate pattern engine in AVAudioEngine (hard, not recommended)

Recommended for EchelonCapture: Option 2 (WKWebView)
- Native Swift for sensors (CoreMotion, WatchConnectivity)
- Embedded WKWebView for Strudel JavaScript runtime
- Local audio playback via Web Audio
- App Store distribution
- Best of both worlds

---

๐Ÿ—๏ธ Updated Architecture with Strudel

Sensor โ†’ Latent โ†’ Music Flow

โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
โ”‚                   FULL SYSTEM ARCHITECTURE                   โ”‚
โ”œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ค
โ”‚                                                              โ”‚
โ”‚  ๐Ÿ“ฑ EchelonCapture (iOS)       ๐ŸŽญ Mocopi Bridge (Python)    โ”‚
โ”‚      โ”œโ”€ iPhone Left (Swift)         โ”œโ”€ hip                  โ”‚
โ”‚      โ”œโ”€ iPhone Right                โ”œโ”€ head                 โ”‚
โ”‚      โ”œโ”€ Apple Watch                 โ”œโ”€ left_hand            โ”‚
โ”‚      โ”œโ”€ AirPods                     โ”œโ”€ right_hand           โ”‚
โ”‚      โ””โ”€ ๐ŸŽต Strudel (WKWebView)      โ”œโ”€ left_foot            โ”‚
โ”‚                                     โ””โ”€ right_foot           โ”‚
โ”‚            โ†“                              โ†“                  โ”‚
โ”‚         POST /api/data               POST /api/data         โ”‚
โ”‚            โ†“                              โ†“                  โ”‚
โ”‚  โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ”‚
โ”‚  โ”‚     cc-mcs Cloud Run Backend (Rust)                   โ”‚ โ”‚
โ”‚  โ”‚     - Sensor fusion (all 10+ devices)                 โ”‚ โ”‚
โ”‚  โ”‚     - LIM-RPS latent physics (25D state)              โ”‚ โ”‚
โ”‚  โ”‚     - Computational Rehearsal (predict future)        โ”‚ โ”‚
โ”‚  โ”‚     - Conductor (musical state machine)               โ”‚ โ”‚
โ”‚  โ”‚     - Pattern-coder (trajectory โ†’ edits)              โ”‚ โ”‚
โ”‚  โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โ”‚
โ”‚                           โ†“                                  โ”‚
โ”‚                    WebSocket stream                          โ”‚
โ”‚                           โ†“                                  โ”‚
โ”‚  โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ”‚
โ”‚  โ”‚     Client (Browser or WKWebView in EchelonCapture)   โ”‚ โ”‚
โ”‚  โ”‚     - Receives latent state (50 Hz)                   โ”‚ โ”‚
โ”‚  โ”‚     - Receives pattern edits (per bar)                โ”‚ โ”‚
โ”‚  โ”‚     - Strudel engine (always-on music runtime)        โ”‚ โ”‚
โ”‚  โ”‚     - Web Audio synthesis (speakers/headphones)       โ”‚ โ”‚
โ”‚  โ”‚     - Optional: Neural Audio Server phrases (GPU)     โ”‚ โ”‚
โ”‚  โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โ”‚
โ”‚                           โ†“                                  โ”‚
โ”‚                      ๐Ÿ”Š MUSIC OUTPUT                         โ”‚
โ”‚                                                              โ”‚
โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜

Computational Rehearsal Loop

Your Body (Moving)
    โ†“
Sensors (iPhone, Mocopi, Watch, AirPods)
    โ†“
cc-mcs Backend (LIM-RPS)
    โ†“
Latent State z(t) [25D: tension, excitement, convergence, ...]
    โ†“
Computational Rehearsal โ†’ Predicted Future z(t+ฮ”)
    โ†“
Conductor + Pattern-Coder
    โ†“
Pattern Edits (symbolic commands)
    โ†“
Strudel Runtime (client-side)
    โ†“
๐ŸŽต Music (adapts to your predicted movement)
    โ†“
You hear music โ†’ adjust your movement
    โ†“
(loop continues)

Key Insight: The system doesn't just react to your current movement - it rehearses your near future and generates music that anticipates where you're going.

---

๐ŸŽฏ EchelonCapture's Role with Strudel

### Before Restructuring
EchelonCapture was:
- Data collection tool
- Real-time visualization (Latent Orb, Trajectory)
- Streaming to cc-mcs
- No music generation (music was in cc-dashboard)

### After Restructuring
EchelonCapture becomes:
- Primary performance app for dancers
- Sensor streaming (iPhone + Watch + AirPods)
- Local Strudel music playback (embedded WKWebView)
- Optional recording for training datasets
- No heavy visualizations (simplified UI)
- Mocopi-aware (shows all 10+ sensors)

Why embed Strudel?
- Dancers want to hear music on-device while performing
- Browser-only approach requires separate laptop/tablet for audio
- WKWebView gives native app experience + JavaScript music engine
- Can still use cc-dashboard for DJ/producer view (full visualization)

---

๐Ÿ“ฑ Redesigned EchelonCapture with Strudel

### Core Purpose
Dancer-focused performance app that streams sensors and plays AI-generated music locally.

### What to REMOVE โŒ
1. Visualization components (no longer needed):
- `PerformanceView.swift` - Latent Orb, Trajectory Overlay
- `VisualizationService.swift` - Real-time graphics
- WebSocket `/visualization` connection
- Embodied Features Panel
- Stats overlay

2. WebSocket for visualization (not needed):
- Remove WS client code
- Keep only HTTP POST for sensor data

### What to KEEP โœ…
1. Core sensor streaming:
- `MotionStreamer.swift` - CoreMotion at 50-100Hz
- `NetworkService.swift` - HTTP POST to cc-mcs
- Device role selection (LEFT, RIGHT, WATCH, HEAD)

2. Session management (optional):
- `SessionManager.swift` - Session lifecycle
- `StorageService.swift` - Local recording
- `AudioRecorder.swift` - Audio capture
- Session markers

3. Background execution:
- `BackgroundManager.swift` - Keep streaming while screen off
- Background audio mode (for Strudel playback)

### What to ADD โœจ
1. Strudel Integration (NEW!):
- Add WKWebView to host Strudel JavaScript engine
- Create `StrudelBridge.swift` - Swift โ†” JS communication
- WebSocket connection to cc-mcs for latent state + pattern edits
- Local audio playback via Web Audio API

2. Simplified Performance UI:
- Clean, minimal interface
- Large "Start Performance" button
- Connection status (sensors + backend + music)
- Audio level meter (visual feedback that music is playing)

3. Multi-sensor awareness:
- Status showing all active sensors (iPhone + Mocopi)
- Total sensor count display
- Backend health indicator

---

๐ŸŽจ Proposed New UI Structure

Tab Structure (Simplified)

โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
โ”‚  EchelonCapture                      โ”‚
โ”œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ค
โ”‚                                       โ”‚
โ”‚  โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”โ”‚
โ”‚  โ”‚  Perform    Record    Settings  โ”‚โ”‚
โ”‚  โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜โ”‚
โ”‚                                       โ”‚
โ”‚  โ•”โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•—  โ”‚
โ”‚  โ•‘   PERFORM TAB (NEW!)          โ•‘  โ”‚
โ”‚  โ• โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•ฃ  โ”‚
โ”‚  โ•‘                               โ•‘  โ”‚
โ”‚  โ•‘  ๐ŸŽต MUSIC: Playing            โ•‘  โ”‚
โ”‚  โ•‘  โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ•‘  โ”‚
โ”‚  โ•‘  โ”‚ โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–“โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘โ–‘    โ”‚ โ•‘  โ”‚
โ”‚  โ•‘  โ”‚ Audio Level Meter       โ”‚ โ•‘  โ”‚
โ”‚  โ•‘  โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โ•‘  โ”‚
โ”‚  โ•‘                               โ•‘  โ”‚
โ”‚  โ•‘  ๐Ÿ“ฑ Sensors: 10 active        โ•‘  โ”‚
โ”‚  โ•‘      โ”œโ”€ iPhone (This): โœ…    โ•‘  โ”‚
โ”‚  โ•‘      โ”œโ”€ iPhone Right: โœ…     โ•‘  โ”‚
โ”‚  โ•‘      โ”œโ”€ Apple Watch: โœ…      โ•‘  โ”‚
โ”‚  โ•‘      โ”œโ”€ AirPods: โœ…          โ•‘  โ”‚
โ”‚  โ•‘      โ””โ”€ Mocopi (6): โœ…       โ•‘  โ”‚
โ”‚  โ•‘                               โ•‘  โ”‚
โ”‚  โ•‘  ๐Ÿ”— Backend: Connected        โ•‘  โ”‚
โ”‚  โ•‘  ๐ŸŽญ Latent: Streaming         โ•‘  โ”‚
โ”‚  โ•‘                               โ•‘  โ”‚
โ”‚  โ•‘  โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ•‘  โ”‚
โ”‚  โ•‘  โ”‚  โ–   STOP PERFORMANCE    โ”‚ โ•‘  โ”‚
โ”‚  โ•‘  โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โ•‘  โ”‚
โ”‚  โ•‘                               โ•‘  โ”‚
โ”‚  โ•‘  Stats: 12,450 frames sent   โ•‘  โ”‚
โ”‚  โ•‘         98 Hz | 85ms latency โ•‘  โ”‚
โ”‚  โ•‘                               โ•‘  โ”‚
โ”‚  โ•šโ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•  โ”‚
โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜

### Perform Tab (NEW - Primary View)
Purpose: Start performance, stream sensors, hear music

Features:
- Large status indicators:
- Music status (Playing/Stopped with audio level meter)
- Sensor count + individual sensor status
- Backend connection health
- Latent stream status
- Single action button:
- "Start Performance" โ†’ starts sensors + Strudel
- "Stop Performance" โ†’ stops everything
- Audio feedback:
- Audio level meter (visual confirmation music is playing)
- Optional: Current section (Groove/Build/Climax/Breakdown)
- Optional: BPM display
- Multi-sensor ecosystem view:
- Shows all active devices (iPhone, Watch, AirPods, Mocopi)
- Clear visual: green checkmarks for active sensors
- Minimal stats (collapsible):
- Frames sent, rate, latency

What's REMOVED from old PerformanceView:
- Latent Orb visualization
- Trajectory overlay
- Embodied features panel
- Complex real-time graphics

### Record Tab (Keep, Simplified)
Purpose: Local session recording for training data

Features (simplified from current):
- Session name
- Start/Stop recording
- Moment markers (good/off/peak/drop)
- Session save

Remove: Heavy UI, complex session browser (move to Settings or separate tab)

### Settings Tab (Enhanced)
Purpose: Configuration + ecosystem status

Features:
- Backend Configuration:
- Backend URL (Cloud Run or localhost)
- Connection test button
- Device Configuration:
- Device role (LEFT, RIGHT, WATCH, HEAD)
- Sensor rate (50/100 Hz)
- Strudel Configuration (NEW):
- Audio volume
- Enable/disable neural audio phrases (optional)
- Music style presets (House, Techno, Ambient, etc.)
- Sensor Ecosystem (NEW):
- View all registered devices from backend (GET /devices)
- Mocopi connection status
- Total sensor count
- Refresh button

---

๐Ÿ”ง Implementation Plan

### Phase 1: Remove Visualizations (1-2 hours)
Goal: Strip out all visualization code

Tasks:
1. Delete `PerformanceView.swift`
2. Delete `VisualizationService.swift`
3. Remove WebSocket `/visualization` code from `NetworkService.swift`
4. Simplify `MainView.swift` tabs

Files to modify:
- `Views/MainView.swift` - Remove PerformanceView tab
- `Services/NetworkService.swift` - Remove WebSocket code
- Delete `Views/PerformanceView.swift`
- Delete `Services/VisualizationService.swift`

Deliverable: App compiles, visualizations gone, sensor streaming still works

---

### Phase 2: Add Strudel Integration (4-6 hours)
Goal: Embed Strudel music engine using WKWebView

Tasks:

1. Create Strudel HTML bundle (1 hour):
- Create `strudel-bundle.html` with:
- Strudel JavaScript library (or MotionStrudel variant)
- Tone.js for Web Audio
- WebSocket client for latent state
- Pattern edit interpreter
- Base musical templates (House, Techno, Ambient)

2. Add WKWebView to app (1 hour):
- Create `StrudelView.swift` - SwiftUI view with WKWebView
- Load `strudel-bundle.html` from app bundle
- Handle Web Audio unlock (user gesture required)

3. Create Swift โ†” JS bridge (2 hours):
- Create `StrudelBridge.swift`:
- `startMusic()` - unlock audio context, start Strudel
- `stopMusic()` - stop Strudel engine
- `setVolume(level: Float)` - control audio
- `sendLatentState(data: Data)` - forward latent to Strudel
- `sendPatternEdit(edit: String)` - forward edits to Strudel
- Use WKWebView `evaluateJavaScript()` and `postMessage()`

4. Add WebSocket client (1-2 hours):
- Create `LatentStreamService.swift`:
- WebSocket connection to cc-mcs Cloud Run
- Subscribe to latent state stream (50 Hz)
- Subscribe to pattern edit stream (per bar)
- Forward latent โ†’ Strudel via `StrudelBridge`
- Forward edits โ†’ Strudel via `StrudelBridge`

Files to create:
- `Resources/strudel-bundle.html` - Strudel JavaScript engine
- `Views/StrudelView.swift` - WKWebView container
- `Services/StrudelBridge.swift` - Swift โ†” JS communication
- `Services/LatentStreamService.swift` - WebSocket client

Files to modify:
- `Info.plist` - Add WKWebView permissions
- `Views/MainView.swift` - Add Perform tab with StrudelView

Deliverable: Tap "Start Performance" โ†’ sensors stream, music plays locally

---

### Phase 3: Create Perform UI (2-3 hours)
Goal: Build clean, dancer-focused performance interface

Tasks:
1. Create `PerformView.swift` (new):
- Large "Start/Stop Performance" button
- Audio level meter (read from Strudel via bridge)
- Sensor status grid (iPhone, Watch, AirPods, Mocopi)
- Backend connection indicator
- Stats (collapsible)

2. Add multi-sensor status component:
- Create `MultiSensorStatus.swift`:
- Fetch devices from GET /devices
- Show checkmarks for active sensors
- Group by type (iPhones, Mocopi, Watch, AirPods)

3. Wire up state management:
- Create `PerformViewModel.swift`:
- `isPerforming: Bool`
- `musicPlaying: Bool`
- `activeSensors: [DeviceInfo]`
- `backendConnected: Bool`
- Start/stop logic

Files to create:
- `Views/PerformView.swift` - Main performance UI
- `Views/Components/MultiSensorStatus.swift` - Sensor grid
- `ViewModels/PerformViewModel.swift` - State management

Deliverable: Clean UI, all status indicators working

---

### Phase 4: Simplify Record + Settings (1-2 hours)
Goal: Streamline existing features

Tasks:
1. Simplify `RecordView.swift`:
- Remove complex UI
- Keep essentials: name, start/stop, markers

2. Enhance `SettingsView.swift`:
- Add "Strudel Configuration" section
- Add "Sensor Ecosystem" section (shows all devices)
- Add Mocopi status indicator

Files to modify:
- `Views/RecordView.swift` - Simplify
- `Views/SettingsView.swift` - Add sections

Deliverable: Settings shows Strudel config + ecosystem status

---

### Phase 5: Testing (2-3 hours)
Goal: Verify everything works end-to-end

Tasks:
1. Test sensor streaming:
- Start app, verify POST /api/data works
- Check cc-mcs backend logs for sensor data

2. Test Strudel playback:
- Tap "Start Performance"
- Verify Web Audio unlocks
- Verify music plays from device speakers

3. Test latent streaming:
- Move phone/body
- Verify music reacts to motion
- Check audio level meter responds

4. Test pattern edits:
- Perform large movements (trigger builds/drops)
- Verify musical structure changes

5. Test multi-sensor awareness:
- Run Mocopi bridge (test mode)
- Verify app shows all 10 sensors

6. Test background mode:
- Lock screen while performing
- Verify music continues playing
- Verify sensors keep streaming

Deliverable: Fully working performance app

---

๐Ÿ“Š Before/After Comparison

AspectBeforeAfter
PurposeData collection + visualizationPerformance app (sensors + music)
UI TabsStream, Record, Performance, SettingsPerform, Record, Settings
MusicNone (use cc-dashboard)Strudel embedded (local playback)
VisualizationHeavy (latent orb, trajectory)Minimal (audio meter, status)
Code Size~3,000 lines~2,500 lines (Strudel adds ~500, removes ~1,000)
DependenciesSwiftUI + CoreMotion + AVFoundationSwiftUI + CoreMotion + WKWebView + WebSocket
User FlowStart streaming โ†’ open browser for musicStart performance โ†’ music plays locally
Sensor SupportiPhone + Watch + AirPodsiPhone + Watch + AirPods + Mocopi-aware
BackendPOST /api/data onlyPOST /api/data + WS latent stream
Primary UserData collectorsDancers/Performers

---

๐ŸŽฏ Three-App Ecosystem Strategy (Confirmed)

### App 1: EchelonCapture (This App)
Purpose: Live performance - sensors + music generation
Users: Dancers, performers
Sensors: iPhone, Watch, AirPods (complements Mocopi)
Music: Strudel (embedded WKWebView) + optional Neural Audio Server
Backend: cc-mcs Cloud Run (LIM-RPS + Computational Rehearsal)
Output: Real-time AI-generated music played locally

Keep as separate app because: Primary performance tool, different UX (dancer-focused), music generation core feature

---

### App 2: cc-handguard
Purpose: Behavioral intervention (nail-biting detection)
Users: People with nail-biting habit
Sensors: Apple Watch (wrist motion)
Music: None
Backend: cc-mcs Cloud Run (same backend!)
Output: Haptic intervention when nail-biting detected

Keep as separate app because: Completely different UX (health/behavior), no music, Watch-first sensor strategy

---

### App 3: TrajectoryOS
Purpose: Life trajectory optimization
Users: People optimizing personal growth
Sensors: None (uses backend state)
Music: None
Backend: TrajectoryCore (different backend!)
Output: AI recommendations for life actions

Keep as separate app because: Different domain (life optimization vs motion), different backend, no sensors, no music

---

๐ŸŽผ Where Does Music Generation Live?

Answer: Two places, serving different users:

1. EchelonCapture (iOS): Embedded Strudel for dancers
- WKWebView running Strudel JavaScript
- Local audio playback (iPhone speakers/headphones)
- Simplified UI (just performance controls)
- For: Dancers who want to hear music while moving

2. cc-dashboard (Web): Full Strudel + visualization for DJs/producers
- Browser-based Strudel engine
- Desktop/laptop speakers (better for clubs)
- Full visualization (trajectory, latent state, conductor decisions)
- Producer controls (section overrides, style changes, neural audio toggle)
- For: DJs, producers, researchers, live shows

Both can run simultaneously:
- Dancer uses EchelonCapture on iPhone (hears music in headphones)
- DJ uses cc-dashboard on laptop (controls music for audience)
- Both receive same latent state from cc-mcs backend
- DJ can override or augment dancer's music generation

---

โœ… Decision Matrix: Where Should Strudel Run?

### Option A: Browser only (cc-dashboard) โŒ
Pros: Single codebase, easier maintenance
Cons:
- Dancers need separate device for music (phone for sensors + laptop for audio = awkward)
- Not mobile-friendly for performers
- Requires WiFi/tethering for audio playback

### Option B: WKWebView in EchelonCapture โœ… (RECOMMENDED)
Pros:
- Dancers hear music on-device (iPhone speakers/headphones)
- Native app experience (App Store, background mode)
- Same Strudel JavaScript codebase (shared with cc-dashboard)
- Can still use cc-dashboard for DJ/producer view
Cons:
- WKWebView adds complexity
- Need Swift โ†” JS bridge
- iOS Web Audio has unlock requirement (minor)

### Option C: Native Swift reimplementation โŒ
Pros: Maximum control, no Web Audio limitations
Cons:
- Massive effort (rebuild entire pattern engine in Swift)
- Duplicate codebase (Swift + JavaScript)
- Ongoing maintenance burden
- Lose Strudel's declarative pattern language

DECISION: Option B (WKWebView in EchelonCapture) โœ…

---

๐Ÿš€ Implementation Timeline

### Week 1: Core Restructuring
- [ ] Phase 1: Remove visualizations (1-2h)
- [ ] Phase 2: Add Strudel integration (4-6h)
- [ ] Phase 3: Create Perform UI (2-3h)
- [ ] Phase 4: Simplify Record + Settings (1-2h)
- [ ] Phase 5: Testing (2-3h)

Total: ~10-16 hours (2-3 days)

### Post-Mocopi Hardware Arrival
- [ ] Test with Mocopi bridge running
- [ ] Verify 10-sensor ecosystem (2 iPhone + 1 Watch + 1 AirPods + 6 Mocopi)
- [ ] Test Computational Rehearsal โ†’ Strudel pattern edits
- [ ] Test with Neural Audio Server (GPU phrases)

---

๐Ÿ“ Files to Modify/Create

### Delete
- `EchelonCapture/Views/PerformanceView.swift`
- `EchelonCapture/Services/VisualizationService.swift`

### Create New
- `EchelonCapture/Resources/strudel-bundle.html` - Strudel JS engine
- `EchelonCapture/Views/StrudelView.swift` - WKWebView container
- `EchelonCapture/Views/PerformView.swift` - Main performance UI
- `EchelonCapture/Services/StrudelBridge.swift` - Swift โ†” JS bridge
- `EchelonCapture/Services/LatentStreamService.swift` - WebSocket client
- `EchelonCapture/Views/Components/MultiSensorStatus.swift` - Sensor grid
- `EchelonCapture/ViewModels/PerformViewModel.swift` - State management
- `EchelonCapture/Models/LatentState.swift` - 25D latent data model
- `EchelonCapture/Models/PatternEdit.swift` - Pattern edit commands

### Modify
- `EchelonCapture/Views/MainView.swift` - Add Perform tab
- `EchelonCapture/Views/RecordView.swift` - Simplify
- `EchelonCapture/Views/SettingsView.swift` - Add Strudel config
- `EchelonCapture/Services/NetworkService.swift` - Keep HTTP POST, remove WS visualization
- `EchelonCapture/Info.plist` - Add WKWebView permissions, background audio

---

๐ŸŽฏ Success Criteria

### EchelonCapture is successfully restructured when:
1. โœ… All visualization code removed (PerformanceView, VisualizationService)
2. โœ… Strudel music engine embedded via WKWebView
3. โœ… "Start Performance" button starts sensors + music
4. โœ… Music plays locally from iPhone speakers/headphones
5. โœ… Music reacts to body movement in real-time
6. โœ… Pattern edits from Conductor change musical structure
7. โœ… App shows all active sensors (iPhone + Mocopi + Watch + AirPods)
8. โœ… Background mode works (screen lock doesn't stop music)
9. โœ… No errors, compiles cleanly
10. โœ… Works with cc-mcs backend (POST /api/data, WS latent stream)

---

๐ŸŽญ Computational Rehearsal Integration

How It Works in EchelonCapture

Data Flow:

1. iPhone sensors (100Hz) โ†’ POST /api/data โ†’ cc-mcs backend
2. cc-mcs: LIM-RPS โ†’ 25D latent state z(t) (50Hz)
3. cc-mcs: Computational Rehearsal โ†’ predicted future z(t+ฮ”)
4. cc-mcs: Conductor โ†’ pattern edits based on predicted trajectory
5. cc-mcs โ†’ WebSocket โ†’ EchelonCapture (latent + edits)
6. EchelonCapture: StrudelBridge โ†’ forward to Strudel (WKWebView)
7. Strudel: Apply edits to pattern graph, modulate synth params
8. Web Audio: Output music to iPhone speakers
9. Dancer hears music โ†’ adjusts movement (loop continues)

Key Concepts:
- Latent State (25D): tension, excitement, convergence, divergence, energy, velocity, etc.
- Rehearsal: System predicts where your body is going (1-2 seconds ahead)
- Pattern Edits: Symbolic commands like "increase hi-hat density", "drop bass", "add fill"
- Conductor: Musical state machine (Intro/Groove/Build/Climax/Breakdown/Outro)

Strudel receives:
- Fast stream (50Hz): Current latent values โ†’ modulate continuous params (filter, gain, swing)
- Slow stream (per bar): Pattern edits โ†’ change musical structure at beat boundaries

---

๐Ÿ“š Related Documentation

  • [MOCOPI_README.md](MOCOPI_README.md) - Mocopi integration overview
  • [MOCOPI_ARCHITECTURE_DIAGRAM.md](MOCOPI_ARCHITECTURE_DIAGRAM.md) - System architecture
  • [apps/ios/EchelonCapture/cc-rehearsal.md](apps/ios/EchelonCapture/cc-rehearsal.md) - Computational Rehearsal + Strudel integration theory
  • [apps/ios/EchelonCapture/README.md](apps/ios/EchelonCapture/README.md) - Current app docs
  • [apps/ios/cc-handguard/README.md](apps/ios/cc-handguard/README.md) - HandGuard app
  • [apps/ios/TrajectoryOS/README.md](apps/ios/TrajectoryOS/README.md) - TrajectoryOS app

---

๐Ÿค” Open Questions for User

1. Strudel HTML Bundle: Should we create a custom `strudel-bundle.html` or use existing Strudel library + custom wrapper?
- Recommendation: Start with minimal custom bundle (Strudel + Tone.js + WebSocket client)

2. Music Styles: What musical templates should we include? (House, Techno, Ambient, etc.)
- Recommendation: Start with House (4/4, kick/hats/bass/pads), add more later

3. Neural Audio Server: Should EchelonCapture support GPU-generated phrases, or Strudel only at first?
- Recommendation: Strudel only initially, add Neural Audio later (Phase 2)

4. Recording Features: Keep local recording, or is EchelonCapture purely for live performance?
- Recommendation: Keep recording for training data collection (useful for Computational Rehearsal)

5. Dual Deployment: Should cc-dashboard also embed Strudel for DJ use?
- Recommendation: Yes - same Strudel codebase, different UI (producer controls vs dancer controls)

---

๐ŸŽญ Summary

Key Insight: With Strudel music generation and Mocopi sensors, EchelonCapture transforms from a data collection tool into a complete performance app.

New Role: Dancer-focused app that:
- Streams iPhone/Watch/AirPods sensor data
- Plays AI-generated music locally (Strudel in WKWebView)
- Reacts to your movement via Computational Rehearsal
- Shows multi-sensor ecosystem status (Mocopi awareness)
- No visualizations (simplified UI)
- Real-time musical performance (body โ†’ music in <300ms)

Architecture:
- Sensors: Native Swift (CoreMotion, WatchConnectivity)
- Music: Strudel JavaScript (WKWebView + Web Audio)
- Brain: cc-mcs Cloud Run (Rust LIM-RPS + Computational Rehearsal)
- Communication: HTTP POST (sensors) + WebSocket (latent + edits)

Three-App Strategy: Keep apps separate by domain:
- EchelonCapture: Performance (sensors + music)
- cc-handguard: Behavioral intervention
- TrajectoryOS: Life optimization

Strudel Deployment:
- EchelonCapture (iOS): Embedded WKWebView for dancers
- cc-dashboard (Web): Full browser version for DJs/producers

Next Step: Begin Phase 1 (Remove Visualizations) and Phase 2 (Add Strudel Integration) once user approves plan.

---

Version: 2.0
Status: โœ… Ready for review (updated with Strudel integration)
Estimated Work: 10-16 hours (2-3 days)

---

Built for the multi-sensor Computational Choreography ecosystem with Strudel music generation ๐ŸŽญ๐ŸŽต

Promotion Decision

Attach run IDs, datasets, metrics, and reproduction commands.

Source Anchor

projects/Documentation/_archive/2024-12/ECHELON_CAPTURE_RESTRUCTURING.md

Detected Structure

Method ยท Evaluation ยท Code Anchors ยท Architecture