Grand Diomande Research · Full HTML Reader

LUME Full System Architecture -- K11 as Central Hub

_Generated 2026-04-27 via Evo3 (4-stage recursive creative evolution)._ _Full evolution output: Desktop/evo-cube-output/lume-full-system-architecture/_

Embodied Trajectory Systems architecture technical paper candidate score 58 .md

Full Public Reader

LUME Full System Architecture -- K11 as Central Hub

_Generated 2026-04-27 via Evo3 (4-stage recursive creative evolution)._
_Full evolution output: Desktop/evo-cube-output/lume-full-system-architecture/_

---

Architecture Decision: Two-Plane, Three-Domain

Two communication planes:
- Data Plane (UDP, real-time, <50ms): Sensor data and derived features flow as LUME wire format datagrams. Publishers bind [ip]. Fire-and-forget. No broker.
- Control Plane (HTTP/WS, 100ms+ tolerance): Session management, director cuts, echelon state. Reliable delivery via multicam-server :9404.

Three sovereignty domains:
- K11 owns visual rendering (sensors -> Unity -> HDMI)
- MotionMix iOS owns musical intelligence (128D canonical -> SAN -> Strudel)
- Mac1 multicam-server owns coordination (director, sessions, beat quantizer)

No machine crosses its sovereignty boundary. K11 does not make music decisions. iOS does not control visuals. Mac1 does not process sensor data.

---

Complete Data Flow Diagram

+============================================================================+
|                        LUME FULL SYSTEM ARCHITECTURE                        |
+============================================================================+

+--[ K11 Pod (Windows 11, Ryzen 9 8945HS, Radeon 780M, 32GB) ]-----------+
|                                                                           |
|  HARDWARE SENSORS                                                         |
|  +------------------+  +------------------+  +------------------+         |
|  | Femto Bolt       |  | UMA-8 Mic Array  |  | Sony mocopi      |         |
|  | USB-C, 640x576   |  | USB-A, 8ch 48kHz |  | 6x BLE IMU      |         |
|  | 30fps ToF depth  |  |                  |  | via Sony PC app  |         |
|  +--------+---------+  +--------+---------+  +--------+---------+         |
|           |                      |                      |                 |
|  PYTHON PUBLISHERS (NSSM services, pre-login auto-start)                  |
|  +--------v---------+  +--------v---------+  +--------v---------+        |
|  | pointcloud_pub.py|  | audio_pub.py     |  | mocopi_bridge.py |        |
|  | --raw-depth      |  | --mic --ch 8     |  | :12351 -> LUMM   |        |
|  | LUMD :9700       |  | LUMF :9701       |  | LUMM :9702       |        |
|  | 40B hdr+716px/ch |  | 84B fixed, 60Hz  |  | 772B fixed, 50Hz|        |
|  +--------+---------+  +--------+---------+  +--------+---------+        |
|           |                      |                      |                 |
|           +----------> bind [ip] (Tailscale peers also receive) <-----+
|           |                      |                      |                 |
|  UNITY 6000.3.2f1 + URP 17 + VFX Graph 17 (Vulkan/DX12)                  |
|  +-------------------------------------------------------------------+   |
|  |                                                                     |   |
|  |  INPUT            [100] LumeUdpReceiver      <-- :9700 LUMD        |   |
|  |                   [200] LumeAudioFftReceiver  <-- :9701 LUMF        |   |
|  |                   [100] LumeMocopiReceiver    <-- :9702 LUMM  *NEW* |   |
|  |                   [100] LumeEchelonReceiver   <-- :9704 LUMH  *NEW* |   |
|  |                                                                     |   |
|  |  PROCESSING       [0]   LumeDepthReprojector  (GPU pinhole)        |   |
|  |                   [0]   LumeOpticalFlow       (LK dense flow)      |   |
|  |                   [150] LumeMotionGate        (_LumeTimeScale)     |   |
|  |                                                                     |   |
|  |  VISUAL OUTPUT    [0]   LumePointRenderer     (instanced draw)     |   |
|  |                   [0]   LumeAudioReactor      (flash/jitter)       |   |
|  |                   [210] LumeVfxRuntimeBridge   (-> VFX Graph)      |   |
|  |                   [220] LumeTransientForcePusher (beat impulse)    |   |
|  |                                                                     |   |
|  |  NETWORK OUTPUT   [250] LumeSystemBus         --> :9703 LUMS *NEW* |   |
|  |                                                                     |   |
|  |  --> URP 17 --> HDMI OUT --> DISPLAY                                |   |
|  +-------------------------------------------------------------------+   |
|                                                                           |
|  RELAY               echelon_relay.py  *NEW*                              |
|                      WS :9404/director/ws --> LUMH :9704                  |
+---------------------------------------------------------------------------+
     |              |              |              |
     | LUMD :9700   | LUMF :9701   | LUMM :9702   | LUMS :9703
     | (local only) | (local+mesh) | (local+mesh) | (mesh only)
     |              |              |              |
     v              v              v              v
                    Tailscale Mesh
                         |
     +-------------------+-------------------+
     |                                       |
     v                                       v
+--[ MotionMix iOS ]------------------+  +--[ Mac5 ]-------------+
|                                      |  |                       |
|  RECEIVERS                           |  | Strudel.js :9600      |
|  LummReceiver    <-- :9702 LUMM *NEW*|  | Receives SAN params   |
|  LumeAudioRx    <-- :9701 LUMF *NEW*|  | from iOS at 10Hz      |
|  LumsReceiver    <-- :9703 LUMS *NEW*|  | Renders audio         |
|                                      |  | -> speakers / line out|
|  LOCAL SENSORS                       |  +----------^------------+
|  CoreMotion (60Hz, accel/gyro/att)   |             |
|  Vision BodyPose (30Hz, landmarks)   |             |
|  Camera -> LiveStreamServer :8081    |             |
|                                      |             |
|  128D CANONICAL VECTOR               |             |
|  [0:75]   Echelon (Rust FFI)        |             |
|  [75]     modality mask              |             |
|  [76:100] MocopiExtractor (LUMM)    |             |
|  [100:102] Pocket IMU               |             |
|  [102:104] Watch                    |             |
|  [104:108] LUMF ambient audio *NEW* |             |
|  [108:116] LUMS scene state  *NEW*  |             |
|                                      |             |
|  SAN Pipeline (30Hz)                 |             |
|  FAN->FuseMoE->NHA->TTT->FiLM      |             |
|  -> bpm_target, energy_target       |             |
|  -> pattern_intensity, variation    |--> StrudelWebEngine
|                                      |     POST /san (2Hz)
|  ParamMapper (30Hz)                  |     WKWebView (10Hz)
|  -> brightness, density, energy     |
|  -> cadenceHz, rhythmStability      |
|                                      |
|  POST :9404/echelon (250ms)         |
|  POST :9404/bar-fire (on bar)       |
+--------------------------------------+

+--[ Mac1 ]-------------------------------+
| multicam-server :9404 (Rust/Axum)        |
| Director loop (250ms tick)               |
| EchelonState, BeatQuantizer             |
| WebSocket: /director/ws -> echelon_relay |
| Session: /session/start, /session/end   |
| Stills: /capture, /proof/*              |
+------------------------------------------+

+--[ cloud-vm ]---------------------------+
| Diffusion inference (CUDA, REST API)     |
| Async (200ms+), not real-time path       |
| Called by Mac5 or Mac1                   |
+------------------------------------------+

---

Data Flow Paths (every hop labeled)

Path A: Depth -> Visuals (K11 local, ~13ms)

Femto Bolt (USB-C) -> pyorbbecsdk (pointcloud_pub.py)
  -> LUMD :9700 (loopback, 40B hdr, 716px/chunk, 30fps)
  -> LumeUdpReceiver (magic dispatch)
  -> LumeDepthReprojector (GPU compute shader, pinhole model)
  -> LumeOpticalFlow (frame-diff + LK dense flow)
  -> LumeMotionGate (motion -> _LumeTimeScale)
  -> LumePointRenderer -> URP 17 -> HDMI

Path B: Audio -> Visual Reactivity (K11 local, ~8ms)

UMA-8 (USB-A) -> sounddevice (audio_pub.py --mic)
  -> LUMF :9701 (loopback, 84B fixed, 60fps)
  -> LumeAudioFftReceiver (RMS, 4 bands, transient bit)
  -> _AudioLevels, _OutlineFlash, _InnerSpread globals
  -> LumeAudioReactor + LumeTransientForcePusher
  -> LumeVfxRuntimeBridge -> VFX Graph particles

Path C: Mocopi -> Visuals (K11 local, ~20ms)

Sony mocopi (6x BLE) -> Sony PC app (K11)
  -> :12351 (localhost, Sony TLV binary)
  -> mocopi_bridge.py (parse + reformat)
  -> LUMM :9702 (loopback, 772B fixed, 50Hz)
  -> LumeMocopiReceiver (27 bone pos+quat)
  -> _MocopiBonePositions[27] shader globals
  -> skeleton-reactive visuals

Path D: Audio -> Music Intelligence (K11 -> iOS, ~25ms)

audio_pub.py publishes LUMF to [ip]:9701
  -> Tailscale (~10ms) -> MotionMix iOS LumeAudioReceiver
  -> 128D[104:108] (rms, centroid, onset, dominant_band)
  -> SAN inference -> music parameter modulation

Path E: Mocopi -> Music Synthesis (K11 -> iOS -> Mac5, ~50ms)

mocopi_bridge.py publishes LUMM to [ip]:9702
  -> Tailscale (~10ms) -> MotionMix iOS LummReceiver
  -> MocopiFeatureExtractor -> 24D features
  -> 128D[76:100] (joint velocities, limb ratios, spatial)
  -> SAN + ParamMapper -> music parameters
  -> StrudelWebEngine -> Mac5 Strudel.js :9600 (~10ms)
  -> Audio render (~8ms) -> speakers

Path F: Phone Sensors -> Music (iOS local, ~5ms)

CoreMotion (accel/gyro/attitude) -> EchelonBridge (Rust FFI)
  -> 128D[0:75] (z latent, velocity, temporal)
Vision BodyPose -> cachedPoseFeatures -> 128D[63:69]
  -> SAN -> ParamMapper -> StrudelWebEngine -> Mac5

Path G: Scene State -> Music (K11 -> iOS, ~15ms) [NEW]

Unity LumeSystemBus (aggregates 12 scene features)
  -> LUMS :9703 (64B fixed, 30Hz)
  -> Tailscale -> iOS LumsReceiver
  -> 128D[108:116] (motion_energy, flow_mag, time_scale, bone_count)
  -> SAN modulation: visual intensity feeds back into music

Path H: Echelon -> Visual Sync (iOS -> Mac1 -> K11, ~100ms)

EchelonBridge.step() -> POST :9404/echelon (existing)
  -> multicam-server EchelonState
  -> WS /director/ws broadcast
  -> echelon_relay.py (K11) subscribes
  -> LUMH :9704 (48B fixed, 10Hz)
  -> LumeEchelonReceiver
  -> _LumeSection, _LumeTension, _LumeTempo globals
  -> Section-triggered preset transitions, bar-synced bursts

Path I: ML Inference (async, 200ms+)

Mac5 or Mac1 HTTP POST -> cloud-vm via Tailscale
  -> Diffusion model (CUDA, one-step)
  -> Style transfer parameters or generated texture
  -> Applied to Unity via config update

---

Computational Choreography: Motion to Music

128D Canonical Vector (assembled on iOS)

DIMENSION   SOURCE                    CONTENT
[0:32]      Echelon Rust FFI          z vector (16D LIM-RPS latent, padded)
[32:64]     Echelon Rust FFI          velocity (16D, padded)
[63:69]     Vision BodyPose           pose spatial features (meanX/Y, stdX/Y, rangeX/Y)
[69:75]     Echelon temporal          tempo, phase, periodicity, grounding, verticality, rotation
[75]        Swift bitfield            modality mask: cam=1, pocket=2, mocopi=4, watch=8
[76:82]     MocopiExtractor (LUMM)    joint velocities (hips, chest, head, hands, feet)
[82:88]     MocopiExtractor (LUMM)    limb ratios (arm ext L/R, leg ext L/R, spine curl, shoulder)
[88:94]     MocopiExtractor (LUMM)    temporal derivatives (acceleration patterns)
[94:100]    MocopiExtractor (LUMM)    spatial features (CoM velocity, spread, asymmetry)
[100:102]   CoreMotion attitude       pocket IMU (pitch, roll)
[102:104]   WatchKit                  watch (HR normalized, wrist energy)
[104:108]   LumeAudioRx (LUMF) *NEW*  ambient audio (rms, centroid_norm, onset, band_dominant)
[108:116]   LumsReceiver (LUMS) *NEW* scene state (motion_energy, flow_mag, time_scale, bone_ct,
                                       audio_rms, transient, preset_id, frame_rate)
[116:128]   reserved                  zeros (future sensors)

Motion-to-Music Pipeline

128D Canonical Vector
        |
        v
+-- SAN Pipeline (30Hz, 135K params) --------------------------------+
|  FAN (Feature-Attention Network)                                     |
|  -> attends to active modalities based on modality mask             |
|                                                                      |
|  FuseMoE (6 experts, top-2 routing)                                 |
|  -> expert specialization: body, rhythm, ambient, scene, gesture    |
|                                                                      |
|  NHA (Neuro-Harmonic Adapter)                                       |
|  -> maps latent to harmonic space                                   |
|                                                                      |
|  TTT (Temporal Texture Transform)                                    |
|  -> smooths over time, prevents jitter                              |
|                                                                      |
|  FiLM Heads (output):                                               |
|    bpm_target          -> Strudel.js tempo                          |
|    energy_target       -> Strudel.js pattern complexity             |
|    gesture_confidence  -> response strength                         |
|    pattern_intensity   -> instrument layer density                  |
|    pattern_variation   -> rhythmic variation                        |
+----------------------------------------------------------------------+
        |
        v
+-- ParamMapper (30Hz, reactive) -------------------------------------+
|  rightHeight * 1.5 + velocity boost  ->  brightness (pads/filter)   |
|  leftHeight  * 1.5 + velocity boost  ->  density (hats/percussion)  |
|  bodyEnergy  * 2.0 + velocity boost  ->  energy (overall)           |
|  filterCutoff = 100 + brightness * 9900 Hz                         |
|                                                                      |
|  Cadence detection:                                                  |
|    peak detection on motion signal -> motionCadenceHz (0-3.4 Hz)    |
|    interval variance -> rhythmStability (0-1)                       |
|                                                                      |
|  Motion accent (impulse trigger):                                    |
|    sum of velocity derivatives across body -> motionAccent (0-1)    |
|    triggers drum fills, transitions                                  |
+----------------------------------------------------------------------+
        |
        v
+-- StrudelWebEngine -> Mac5 Strudel.js :9600 -----------------------+
|  sanUpdate({bpm_target, energy_target, gesture_confidence,          |
|             pattern_intensity, pattern_variation})                    |
|  Strudel modifies live pattern: add/remove instruments,             |
|  change tempo, alter rhythmic complexity                            |
+----------------------------------------------------------------------+
        |
        v
    Audio output -> speakers

Mocopi -> Music (specific mapping)

The 24D MocopiFeatureExtractor output fills 128D[76:100]:

[76:82]   Joint velocities: hips, chest, head, left_hand, right_hand, left_foot
          -> Higher velocity = more energy in SAN output
          -> Hand velocity asymmetry = pattern variation

[82:88]   Limb ratios: arm extension L/R, leg extension L/R, spine curl, shoulder width
          -> Extended arms = higher brightness (ParamMapper right/left arm)
          -> Spine curl = core motion -> body energy boost
          -> Shoulder width change = chest flex detection territory

[88:94]   Temporal derivatives: acceleration of velocities
          -> Sudden acceleration = motion accent -> drum fill trigger
          -> Sustained acceleration = building energy arc

[94:100]  Spatial features: center of mass velocity, spread, asymmetry, grounding
          -> High CoM velocity = dancing, not standing
          -> High spread = expansive movement = fuller sound
          -> Asymmetry = complex, non-symmetric movement = more variation
          -> Grounding = feet planted vs airborne = bass emphasis

---

Echelon Multi-Device Coordination

Existing Mechanism

iOS EchelonBridge.step() (60Hz)
  -> echelon_step() (Rust FFI)
  -> latent state (16D), lexicon (tension, divergence), section (0-6)
  -> POST :9404/echelon (250ms)
  -> multicam-server EchelonState

multicam-server director_loop (250ms tick):
  -> reads EchelonState
  -> detect_director_scoring_mode():
       if echelon.is_fresh() && echelon.speed > 0.15 -> Performance mode
       else -> Creator mode (or Photoshoot if configured)
  -> BeatQuantizer.tick():
       soft_tempo_bpm, bar_phase, bar_count, bar_boundary events
  -> TemporalIntelligence:
       energy arc tracking, adaptive cut timing

New Mechanism for LUME Visual Sync

multicam-server /director/ws broadcast (existing):
  JSON messages including echelon state + bar_fire events

echelon_relay.py (NEW, runs on K11):
  WebSocket client -> :9404/director/ws
  Extracts: section_state, tension, divergence, tempo_bpm, bar_phase,
            energy_arc, bar_count, scoring_mode
  On bar_fire: sets bar_boundary_now flag
  Publishes: LUMH :9704 (48B fixed, 10Hz, loopback to Unity)

Unity LumeEchelonReceiver (NEW):
  Receives LUMH :9704
  Sets shader globals:
    _LumeSection (int)     -> preset auto-switch on section transitions
    _LumeTension (float)   -> visual intensity modulation
    _LumeTempo (float)     -> beat-synced particle effects
    _LumeBarPhase (float)  -> bar-position-aware animation
  On bar_boundary_now:
    -> SendEvent("OnBarBeat") to VFX Graph
    -> flash/burst aligned to musical bars

Beat Sync Accuracy

Bar-fire relay path: iOS -> HTTP :9404 -> WS broadcast -> echelon_relay -> LUMH -> Unity
Estimated latency: ~100ms end-to-end
This is adequate for bar-aligned events (bars = 1-4 seconds)
NOT adequate for beat-aligned events (beats = 200-500ms)
Beat-aligned visual hits use LOCAL LUMF transient detection instead (Path B, ~8ms)
Result: bar transitions are server-coordinated, beat hits are locally detected

---

Where Diffusion/ML Inference Fits

+-- ML Inference Architecture --+
|                                |
|  K11: NO ML inference          |
|  (Radeon 780M, no CUDA)       |
|                                |
|  Mac5: CoreML/MLX inference    |
|  - ConditioningEncoder (104D)  |
|  - Future: 128D retrained      |
|  - One-step diffusion          |
|  - Style generation            |
|  - Latency: 50-200ms           |
|  - Access: REST via Tailscale  |
|                                |
|  cloud-vm: CUDA inference      |
|  - Full diffusion models       |
|  - Batch processing            |
|  - Latency: 200ms+             |
|  - Access: REST via Tailscale  |
|                                |
|  Use cases:                    |
|  1. Visual style transfer:     |
|     Generate texture/palette   |
|     from motion signature      |
|     (async, applied to Unity   |
|     preset via config update)  |
|                                |
|  2. Audio generation:          |
|     One-step audio diffusion   |
|     for unique sound design    |
|     (async, loaded into        |
|     Strudel.js as samples)     |
|                                |
|  3. Training data:             |
|     Generate training pairs    |
|     for SAN V6 retrain         |
|     (offline, batch)           |
+--------------------------------+

---

Role of MotionMix LiveDirector

The multicam-server :9404 LiveDirector serves LUME in three specific ways:

1. Session lifecycle. `/session/start` puts all devices in performance mode. `/session/end` stops. K11's echelon_relay subscribes to session events and can trigger Unity scene changes.

2. Beat coordination. The BeatQuantizer tracks tempo from EchelonState. Bar-fire events propagate to K11 via echelon_relay for bar-aligned visual transitions.

3. Multi-camera recording. If iPhones are filming the LUME performance, the director loop scores camera angles and switches cuts. This runs independently of K11 visuals. The camera footage + LUME visual output can be composited in post.

The LiveDirector does NOT control K11 visuals directly. It provides coordination signals that K11 consumes as one of many inputs.

---

Port Map (complete)

Port	Protocol	Wire Format	Magic	Size	Hz	Producer	Consumer(s)	Plane
9700	UDP	LUMD	0x4C554D44	40B+1432B	30	pointcloud_pub.py (K11)	Unity LumeUdpReceiver (K11)	Data
9700	UDP	LUME	0x4C554D45	16B+variable	30	pointcloud_pub.py --synthetic	Unity LumeUdpReceiver (K11)	Data
9701	UDP	LUMF	0x4C554D46	84B fixed	60	audio_pub.py (K11)	Unity (K11), iOS	Data
9702	UDP	LUMM	0x4C554D4D	772B fixed	50	mocopi_bridge.py (K11)	Unity (K11), iOS	Data
9703	UDP	LUMS	0x4C554D53	64B fixed	30	Unity LumeSystemBus (K11)	iOS	Data
9704	UDP	LUMH	0x4C554D48	48B fixed	10	echelon_relay.py (K11)	Unity LumeEchelonReceiver (K11)	Data
9404	HTTP/WS	JSON	--	variable	--	multicam-server (Mac1)	All devices	Control
9600	HTTP/WS	Strudel	--	variable	--	Strudel.js (Mac5)	iOS (writer)	Control
8081	HTTP	MJPEG	--	variable	10-18	LiveStreamServer (iOS)	Browsers	Control
12351	UDP	Sony TLV	MCS\0	variable	~50	Sony mocopi PC app (K11)	mocopi_bridge.py (K11 localhost)	Data

---

Code Inventory: Exists vs Needs Writing

EXISTS (no changes needed)

Component	Lines	Location
pointcloud_pub.py	556	lume-commerce/software/demo/
audio_pub.py	315	lume-commerce/software/demo/
10 Unity C# components	~2400	Assets/Scripts/
2 compute shaders	~300	Assets/Shaders/
3 editor scripts	~300	Assets/Editor/
5 VFX preset ScriptableObjects	~200	Assets/Scripts/Vfx/
37 Python tests	~800	tests/
EchelonBridge.swift	~400	MotionMixApp/Services/
ParamMapper.swift	218	MotionMixApp/Services/
ChestFlexDetector.swift	330	MotionMixApp/Services/
StrudelWebEngine.swift	236	MotionMixApp/Services/
AudioEngine.swift	~400	MotionMixApp/Services/
LiveStreamServer.swift	392	MotionMixApp/Services/
MocopiFeatureExtractor	~150	MotionMixApp/
multicam-server	10678	MotionMix/multicam-server/
Echelon (20 crates)	~50K	Comp-Core/cc-echelon/
Total existing	~67,000+

NEEDS TO BE WRITTEN

Component	Est. Lines	Location	Purpose
mocopi_bridge.py (LUMM)	~200	lume-commerce/software/demo/	Sony :12351 -> LUMM :9702
LumeMocopiReceiver.cs	~120	Unity Assets/Scripts/	LUMM consumer, bone globals
LumeSystemBus.cs	~100	Unity Assets/Scripts/	LUMS :9703 publisher
LumeEchelonReceiver.cs	~80	Unity Assets/Scripts/	LUMH :9704 consumer
echelon_relay.py	~80	lume-commerce/software/demo/	WS :9404 -> LUMH :9704
LummReceiver.swift	~80	MotionMixApp/Services/	LUMM consumer on iOS
LumsReceiver.swift	~60	MotionMixApp/Services/	LUMS consumer on iOS
LumeAudioReceiver.swift	~40	MotionMixApp/Services/	LUMF consumer on iOS
install-services.ps1	~100	scripts/k11/	NSSM service setup
health-check.ps1	~80	scripts/k11/	Windows diagnostics
firewall-rules.ps1	~20	scripts/k11/	UDP port rules
test_mocopi_bridge.py	~150	tests/	Bridge test suite
test_echelon_relay.py	~80	tests/	Relay test suite
lume-k11.json	~30	config/	Deployment config
Wire format specs (LUMM+LUMS+LUMH)	~80	lume_wire_format.md	Documentation
Total new	~1,300

NEEDS MODIFICATION (small changes)

Component	Change	Est. Lines
lume_packet_inspector.py	Add LUMM+LUMS+LUMH magic	+10
LumeWaveAutoWire.cs	Wire 3 new components	+20
LumeCalibrationPanel.cs	Add mocopi/echelon tunables	+30
EchelonBridge.swift	Wire 3 new receivers	+40
MotionMixApp.swift	Initialize new receivers	+15
Total modified		+115

---

New Wire Format Specifications

LUMM (0x4C554D4D) -- Mocopi Skeleton

Port: `:9702`. Fixed 772 bytes.

struct Header {             // 16 bytes
    u32 magic;              // 0x4C554D4D
    u32 frame;              // monotonic
    u16 bone_count;         // 27
    u16 flags;              // bit0: has_quat, bit1: has_pos, bit2: calibrated
    f32 dt_seconds;
};

struct BoneTransform {      // 28 bytes x 27
    f32 position[3];        // parent-relative xyz
    f32 quaternion[4];      // local rotation xyzw
};

LUMS (0x4C554D53) -- Scene State

Port: `:9703`. Fixed 64 bytes. Published at 30Hz by Unity LumeSystemBus.

struct Header {             // 16 bytes
    u32 magic;              // 0x4C554D53
    u32 frame;
    u16 field_count;        // 12 (v1)
    u16 flags;
    f32 dt_seconds;
};

struct Body {               // 48 bytes (12 x f32)
    f32 motion_energy;      // _InnerSpread
    f32 flow_magnitude;     // optical flow RMS
    f32 time_scale;         // _LumeTimeScale
    f32 audio_rms;          // _AudioLevels.x
    f32 audio_band_sub;
    f32 audio_band_low;
    f32 audio_band_mid;
    f32 transient_active;   // 1.0 on transient frame
    f32 bone_count;         // from LumeMocopiReceiver
    f32 root_velocity;      // _MocopiRootVelocity
    f32 preset_id;          // current VFX preset
    f32 frame_rate;         // 1/deltaTime
};

LUMH (0x4C554D48) -- Echelon/Harmony State

Port: `:9704`. Fixed 48 bytes. Published at 10Hz by echelon_relay.py.

struct {                    // 48 bytes
    u32 magic;              // 0x4C554D48
    u32 frame;
    u8  section_state;      // 0-6
    u8  scoring_mode;       // 0=idle,1=perf,2=creator,3=photo
    u16 flags;              // bit0: bar_boundary_now
    f32 tension;            // 0.0-1.0
    f32 divergence;         // 0.0-1.0
    f32 soft_tempo_bpm;
    f32 bar_phase;          // 0.0-1.0
    f32 energy_arc;         // 0.0-1.0
    f32 dt_seconds;
    u32 bar_count;
    u32 reserved[2];
};

---

Execution Waves (58 tasks, 8 waves, 6-7 day critical path)

Wave	Day	Tasks	Focus	Kill Criterion
0	1	1-9	K11 bootstrap, Tailscale, synthetic test	K11 can't run Python or reach Tailscale
1	2	10-16	Femto Bolt + UMA-8 real sensors	pyorbbecsdk fails on Windows
2	2-3	17-23	Unity on K11, compute shader test	Compute shaders fail on 780M
3	3-4	24-29	LUMM mocopi bridge	Sony parse fails AND OSC fails
4	4-5	30-39	Unity LUMM+LUMS+LUMH components	--
5	5	40-46	MotionMix iOS integration	--
6	5-6	47-51	NSSM services, production config	--
7	6-7	52-58	Integration testing, hardening	--

Promotion Decision

Promote into a technical note or architecture paper with implementation anchors.

Source Anchor

lume-commerce/docs/lume-system-architecture.md

Detected Structure

Method · Evaluation · Figures · Code Anchors · Architecture