Path E: Event-Driven Mesh -- NATS/Pub-Sub Replaces Direct UDP

Path E: Event-Driven Mesh -- NATS/Pub-Sub Replaces Direct UDP

Concept

Replace all direct UDP connections with a NATS JetStream message bus. Every publisher and consumer becomes a NATS client. Topics replace ports: `lume.depth`, `lume.audio`, `lume.skeleton`, `lume.echelon`, `lume.music`, `lume.director`. Dynamic routing: any new consumer subscribes to a topic and immediately gets data. Replay: JetStream persists N seconds of history so a late-joining consumer catches up. Health monitoring: NATS provides built-in consumer lag metrics.

Architecture

NATS JetStream (on K11 or Mac1)
+-----------------------------------------------+
| Subjects:                                      |
|   lume.depth.raw    <- pointcloud_pub.py       |
|   lume.audio.fft    <- audio_pub.py            |
|   lume.skeleton.raw <- mocopi_bridge.py         |
|   lume.echelon.state <- MotionMix EchelonBridge|
|   lume.music.params  <- MotionMix ParamMapper  |
|   lume.director.cuts <- multicam-server         |
|   lume.visual.state  <- Unity LumeSystemBus    |
+-----------------------------------------------+
        ^     ^     ^     ^     ^
        |     |     |     |     |
K11     iOS   Mac5  Mac1  cloud-vm
(pub+sub) (pub+sub) (sub) (pub+sub) (sub)

Strengths

• Dynamic topology. Add a new consumer (e.g., TouchDesigner on a laptop) by subscribing to a topic. No port configuration, no firewall rules.
• Replay/history. JetStream's windowed retention means a late-joining consumer can replay the last N frames. Useful for debugging and development.
• Health monitoring. NATS provides consumer group lag, message rates, and connection health out of the box.
• Decoupled evolution. Change a publisher's format? Just version the subject. Old consumers keep working on the old subject.
• Already partially deployed. Mohamed has NATS JetStream running (see .nats/jetstream/ in git status). MESH_EVENTS and POLICY streams exist.

Weaknesses

• CRITICAL: Latency. NATS adds serialization + deserialization + broker hop. For 84-byte LUMF datagrams at 60Hz, the overhead of NATS framing + TCP + broker routing adds 5-15ms vs direct UDP's <1ms. For LUMD depth at 30fps with 716-pixel chunks, the broker becomes a bottleneck.
• CRITICAL: Throughput. LUMD at 640x576 resolution = 368,640 pixels at 30fps. At 716 pixels/chunk, that's 515 NATS messages per frame, 15,450 messages/second. NATS can handle this, but the chunked depth format was designed for raw UDP sendto, not message broker framing.
• Complexity for no gain on the hot path. The sensor -> Unity visual path is local loopback. Adding NATS between pointcloud_pub.py and Unity on the same machine is pure overhead with zero benefit.
• Dependency. NATS must be running for anything to work. Direct UDP has zero infrastructure dependency.
• Windows NATS server. NATS runs on Windows, but Mohamed's existing NATS setup is on macOS with LaunchAgents. New deployment path.
• Overkill for 3-5 consumers. The system has exactly: Unity (consumer), MotionMix iOS (consumer), Mac5 Strudel (consumer). Three consumers don't need a message broker.

Verdict

REJECT for the hot path (sensor data). Direct UDP is correct for real-time sensor data on the local machine. PARTIAL ADOPT for the control plane: NATS could be useful for coordination messages (director cuts, session state, echelon events) where latency tolerance is higher (100ms+) and dynamic subscription is valuable. But sensor data must stay on direct UDP.