Path C: MotionMix-as-Brain -- iOS App is the Conductor, K11 is Just Sensors

Path C: MotionMix-as-Brain -- iOS App is the Conductor, K11 is Just Sensors

Concept

MotionMix iOS is the master. It already has EchelonBridge (128D canonical vector), ParamMapper, SAN, ChestFlexDetector, MocopiFeatureExtractor, AudioEngine, StrudelWebEngine, and LiveStreamServer. K11 becomes a dumb sensor bridge: it runs publishers and Unity for visual rendering, but all intelligence decisions flow from the iPhone. The phone tells K11 what to render and Mac5 what to play.

Architecture

MotionMix iOS (BRAIN)
+--------------------------------------------------+
| EchelonBridge (60Hz, 128D canonical)              |
| SAN pipeline (30Hz, 135K params)                  |
| ParamMapper (motion -> audio params)              |
| ChestFlexDetector (pec flex -> triggers)          |
| MocopiFeatureExtractor (27 bones -> 24D)          |
| StrudelWebEngine -> Mac5 Strudel (audio)          |
| AudioEngine (fallback synthesis)                  |
|                                                    |
| Consumes: LUMM :9702 from K11 (mocopi skeleton)   |
| Consumes: LUMF :9701 from K11 (audio features)    |
| Publishes: VISUAL_CMD to K11 Unity (NEW protocol) |
| Publishes: MUSIC_PARAMS to Mac5 Strudel           |
+--------------------------------------------------+
        |                       |
        | UDP: visual commands  | HTTP: SAN head state
        v                       v
K11 (Sensor Bridge + Renderer)  Mac5 (Audio Synthesis)
+---------------------------+  +-------------------+
| Femto -> LUMD :9700       |  | Strudel.js :9600  |
| UMA-8 -> LUMF :9701       |  | Receives params   |
| mocopi -> LUMM :9702      |  | Renders audio     |
| Unity: renders visuals    |  +-------------------+
| Follows iOS commands      |
| (color, intensity, mode)  |
+---------------------------+

Strengths

• Leverages the most mature code. EchelonBridge + SAN + ParamMapper is the most tested, most iterated intelligence stack. It's been trained on real data (5,408 pairs).
• iPhone sensor fusion is unmatched. CoreMotion + Vision BodyPose + LiDAR + face detection. No equivalent on K11.
• Already works. The iOS app already drives Strudel.js audio and already consumes mocopi data. This is the smallest delta from current state.
• Single control surface. Mohamed holds the phone. One device to rule them all.

Weaknesses

• CRITICAL: Adds latency to visual path. Depth camera data would need to flow K11 -> iOS -> K11 for any motion-reactive visual decisions. That's a round trip through Tailscale. Unacceptable for 60fps visuals.
• Phone battery. Running EchelonBridge at 60Hz + SAN at 30Hz + network I/O + camera is a battery drain. iPhone 16 Plus has ~4-5 hours of heavy use.
• Phone thermal throttle. Sustained 60Hz compute + camera + networking causes iOS thermal management to kick in after 30-45 minutes.
• Phone is also the camera. If MotionMix is the brain AND a camera node, it's doing too much. The model (Mohamed) holds the phone during performance.
• New protocol needed. "VISUAL_CMD" from iOS to K11 Unity doesn't exist. This is a new wire format, new consumer, new complexity.
• K11 Unity already reacts locally. LumeOpticalFlow, LumeMotionGate, LumeAudioFftReceiver all react to sensor data locally on K11. Making them follow remote commands breaks the local reactivity that makes the visuals feel alive.

Verdict

REJECT as primary architecture. The phone should remain a brain for MUSIC (it already drives Strudel.js) but should NOT be the brain for VISUALS. Visual reactivity must remain K11-local. The phone's correct role is: (1) provide the 128D canonical vector for music synthesis, (2) consume LUMM for MocopiFeatureExtractor, (3) drive Strudel.js audio. It should NOT try to control K11's visual pipeline remotely.