Back to corpus
architecturetechnical paper candidatescore 32

Path F: Live Performance Optimized -- Minimize Latency at All Costs

Every design decision is filtered through one question: does this add latency? If yes, reject it. Direct UDP everywhere. No middleware, no brokers, no HTTP coordination. Publishers bind [ip] and consumers hardcode IPs. The system is static: every connection is pre-configured at deploy time. No dynamic discovery, no subscriptions, no health checks. The absolute minimum number of network hops between any sensor and any output. Where possible, merge publisher and consumer onto the same process.

Open full HTMLDownload HTMLCorpus JSONPaper schema

Full HTML reader

Read the full artifact

Open in new tab

Extracted abstract or opening context

Every design decision is filtered through one question: does this add latency? If yes, reject it. Direct UDP everywhere. No middleware, no brokers, no HTTP coordination. Publishers bind [ip] and consumers hardcode IPs. The system is static: every connection is pre-configured at deploy time. No dynamic discovery, no subscriptions, no health checks. The absolute minimum number of network hops between any sensor and any output. Where possible, merge publisher and consumer onto the same process. | Hop | Path | Latency | |-----|------|---------| | 1 | Femto ToF capture | ~5ms | | 2 | pyorbbecsdk -> Python | ~1ms | | 3 | UDP loopback :9700 | <0.5ms | | 4 | Unity receive + reproject | ~2ms | | 5 | Unity render + display | ~5ms | | **Visual total** | **Sensor -> pixel** | **~13ms** | | | | | | 1 | mocopi BLE -> Sony app | ~15ms | | 2 | Sony -> :12351 -> bridge | ~2ms | | 3 | LUMM loopback :9702 | <0.5ms | | 4 | LUMM Tailscale -> iOS | ~10ms | | 5 | iOS MocopiExtractor | ~1ms | | 6 | iOS SAN + ParamMapper | ~3ms | | 7 | Params Tailscale -> Mac5 | ~10ms | | 8 | Strudel.js render | ~8ms | | **Audio total** | **Skeleton -> sound** | **~50ms** | - **Fastest possible visual response.** ~13ms sensor-to-pixel is essentially the camera's own latency. The processing pipeline adds <8ms. - **Fastest possible audio response.** ~50ms skeleton-to-sound is within the perceptible synchrony window (human perceives audio-visual sync within ~80ms). - **No infrastructure dependencies.** No NATS, no message brokers, no health check services. Just UDP sockets. - **Battle-tested.** This is essentially how the system works today (minus LUMM to iOS). The existing code IS this architecture. - **Rigid topology.** Adding a new consumer means editing publisher code to add a destination. No dynamic discovery. - **No monitoring.** If packets drop, nobody knows. If a publisher crashes, silence is the only signal. - **Hardcoded IPs.** Tailscale IPs are stable but still configuration. If Mac5 changes IP (Tailscale reinstall), manual update. - **Fire-and-forget UDP.** No delivery guarantees. In practice, local loopback is reliable, but Tailscale hops can have 100% loss (documented: Mac5 unreachable during some sessions). - **No coordination layer.** No session start/stop, no state synchronization, no beat clock across machines. Each machine runs independently. - **Music-to-visual sync gap.** K11 visuals react to local sensors. Mac5 audio reacts to iOS-processed sensors. These two reaction paths have different latencies and no shared clock. A transient beat hit on the audio may not visually sync with the depth-triggered impulse. ADOPT as the data plane. Direct UDP is correct for all real-time sensor data. The latency budget is the ground truth that every other architecture must beat or matc

Promotion decision

What has to happen next

Promote into a technical note or architecture paper with implementation anchors.

Why this is not always a full paper yet

Corpus pages are public-safe readers for discovered workspace artifacts. They are not automatically final papers. A corpus item becomes a polished paper only after the editable source, evidence checkpoints, references, figures, render path, and release status are attached through the paper schema.