LUME Multi-Sensor Motion Fusion — Phased Plan

The plan to go from **one camera** (today) to **four fused pose sources**: K11 Femto Bolt + Mac4 Femto Mega + 2 iPhones running MotionMix. The bar opens **single performer, single Bolt**. One camera = one timeline, no fusion, no clock sync required. This entire plan is the post-launch "smarter track." The trigger to begin Phase 1 is: **the bar has launched on the single Bolt and is stable.** Starting earlier is the scope-creep we explicitly cut. | Source | State | |---|---| | K11 Femto Bolt | ✅ Live — pyorbbecsdk COLOR_SENSOR → MediaPipe BlazePose → emits canonical pose JSON on UDP 9705, LUMM 27-bone on 9702, MJPG recording. Verified 2026-05-19. | | Mac4 Femto Mega | ❌ No pose. Orbbec RGB is not a UVC webcam; needs the pyorbbecsdk-color path (same fix already proven on the Bolt). | | iPhone ×2 (MotionMix) | ❌ Not wired. MotionMix does on-device pose but keeps it for its own synth. | **Canonical capture-node contract** (frozen — the Bolt publisher already emits it): `{"frame":int,"t":float,"w":int,"h":int,"landmarks":[{"x","y","z","vis"}×33]}` as UDP datagrams. Every capture node emits this, tagged with a `source` id. **Architecture decisions already locked** (from the 2026-05-19 redundancy audit): - `cc-collection` is the canonical fusion path (1,027-line Kalman engine, parses 33-landmark frames). - `cc-window-aligner` is parked for LUME — real code (20,215 LOC) but mocopi-shaped, its `PacketPayload` has no MediaPipe variant. - `femto-bridge`'s 128D `skeleton_128d::Encoder` is kept — the only 128D code in the stack.