Computational Choreography — System Overview

Multi-modal input streams sampled at 100 Hz: - **Mocopi suit** — 24-point skeleton tracking - **Polar H10** — heart rate + HRV - **Video pose** — MediaPipe/OpenPose fallback - **Mobile accel** — phone/tablet backup Features extracted per frame: - Motion energy (kinetic sum) - HR derivative (arousal) - Spatial symmetry (left/right balance) - Center of mass trajectory - Velocity peaks - Limb angles Converts multi-channel features into a **coherent latent representation** `x⁺`: - Handles missing modalities through hallucination - Maintains temporal coherence via proximal update - Outputs normalized latent vector (dim=16-32) Sliding window (2-4 seconds) of latent vectors → control signals: - `tempo_scale` — BPM multiplier (0.75–1.5) - `density` — rhythmic complexity (0–1) - `tension` — filter/harmonic tension (0–1) - `brightness` — spectral brightness (0–1) - `fx_send` — reverb/delay mix (0–1) - `swing` — rhythmic swing (0–0.6) - `accent` — dynamic range (0–1) Control signals → sound engine protocols: - **MIDI Bridge** — CC messages via Web MIDI (IAC Driver) - **OSC Bridge** — UDP to SuperCollider/Max - **Link Bridge** — Ableton Link sync for phase/tempo