Grand Diomande Research · Full HTML Reader

LUME

Embodied-light visual engine. Multi-machine pipeline that drives a 1920×440 bar display from real-time sensor input — depth, audio, motion — with a music-generation feedback loop.

Embodied Trajectory Systems technical note experiment writeup candidate score 32 .md

Full Public Reader

LUME

Embodied-light visual engine. Multi-machine pipeline that drives a 1920×440 bar display from real-time sensor input — depth, audio, motion — with a music-generation feedback loop.

Repository layout

lume-commerce/
├── apps/                       Compiled binaries (Rust)
│   ├── lume-rust-commerce/     POS / kitchen / voice / payment / NLU
│   └── lume-daemon/            On-device sensor broker + mesh slot
│
├── services/                   Autonomous Python publishers (UDP wire format)
│   ├── audio-pub/              LUMF :9701 — UMA-8 mic FFT + onset
│   ├── depth-pub/              LUMD :9700 — Femto Bolt/Mega raw depth
│   ├── mocopi-bridge/          LUMM :9702 — real Sony Mocopi → LUMM
│   ├── mocopi-synth/           LUMM :9702 — synthetic skeleton (no hardware)
│   └── sony-bridge/            WS:9400 → LUMM:9702 upstream forwarder
│
├── packages/                   Shared libraries
│   └── lume-tools/             packet inspector, calib, replay, probes
│
├── viz/                        Visual stack
│   └── lume-pcloud/            Unity 6000.3.2f1 + URP 17 + VFX Graph
│
├── infra/                      Deployment + service definitions
│   ├── launchagents/           macOS LaunchAgent plists (Mac1, Mac5)
│   └── nssm/                   Windows NSSM service bundle (K11)
│
├── hardware/                   Physical product
│   ├── cad/                    OpenSCAD parametric models + slicer assets
│   ├── design/                 Industrial design references
│   └── reference/              Inspiration corpus (Duncan Fewkes archive)
│
├── docs/                       Specs, runbooks, research
│   ├── architecture/           High-level architecture + mesh integration
│   ├── handoffs/               Codex/Mac5 handoff trail
│   ├── legacy/                 Pre-restructure README archive
│   ├── playbooks/              Duncan v3 + operational runbooks
│   ├── protocols/              Wire format specs (LUME/LUMD/LUMF/LUMM/LUMC)
│   └── research/               Gap analyses, reel corpus, Duncan v3
│
├── scripts/                    Operational scripts
│   ├── bringup/                Per-machine bringup (Mac4/Mac5/Beelink/Jetson)
│   ├── duncan/                 Duncan corpus refresh + library builders
│   └── k11/                    K11 NSSM control scripts
│
└── tests/                      Test suites
    └── services/               Per-service Python tests (pytest)

Wire formats

All UDP, all on loopback unless cross-machine routing is explicitly enabled.

Magic	Port	Source	Sink	Spec
`LUME`	9700	depth-pub (cloud mode)	viz/lume-pcloud	docs/protocols/wire_format_v1.md
`LUMD`	9700	depth-pub (raw-depth)	viz/lume-pcloud	docs/protocols/wire_format_v1.md
`LUMF`	9701	audio-pub	viz/lume-pcloud	docs/protocols/wire_format_v1.md
`LUMM`	9702	mocopi-{bridge,synth,sony-bridge}	viz/lume-pcloud	docs/protocols/wire_format_v1.md
OSC `/lume/...`	9050	viz/lume-pcloud → audio engine	external	(motion-driven music conditioning)

Golden-bytes regression coverage in `tests/services/test_wire_format_golden.py`.

Quick start

Synthetic demo (no hardware)

bash

# In three panes:
python3 services/audio-pub/audio_pub.py --synthetic --bpm 120
python3 services/mocopi-synth/mocopi_synth.py --motion dance --sync-to-lumf
python3 services/depth-pub/depth_pub.py --synthetic-depth
# Inspect packets:
python3 packages/lume-tools/packet_inspector.py
# Open Unity at viz/lume-pcloud/ and Play.

Real-hardware demo

bash

# Mac5 / K11 — Femto Bolt
python3 services/depth-pub/depth_pub.py --raw-depth   # auto-detects Orbbec SDK
# UMA-8 mic on the same host
python3 services/audio-pub/audio_pub.py --device "UMA-8"
# Sony Mocopi bridge (after sensor-relay.js is feeding WS :9400)
python3 services/sony-bridge/sony_to_lumm_bridge.py

Tests

bash

# Python service tests (fast)
cd tests/services && pytest -q
# Slow integration tests (subprocess + sockets)
cd tests/services && LUME_RUN_SLOW=1 pytest -q
# Rust workspace
cargo test --workspace

Baseline: 139 passed / 6 skipped (fast suite).

Machines

Host	Role	Stack
Mac1	dev / canonical / git	full repo + Codex + Claude Code
Mac4	Unity host	viz/lume-pcloud Editor + Play mode
Mac5	ML / fine-tune (rate-limited)	LaunchAgent audio-pub + ML training
K11	production box (Windows)	NSSM services for audio/mocopi/depth
Jetson Orin	(deferred)	ARM64 product target

Don't-touch list

Preserved across all restructure work:
- Wire format magics: `LUME` / `LUMD` / `LUMF` / `LUMM` / `LUMC` (additive only).
- Unity component public APIs (additive only).
- K11 NSSM service definitions (already production at C:\lume\).
- Wave 8 / Wave 9 visual ladder commits (8fb6fe75…83b565e3 in old SHAs; rewritten under post-extraction history).
- `LumeMocopiReceiver.cs` (Mac1 thread-safe version is canonical).

Authorship

Mohamed Diomande — `[email]`. Built collaboratively with Claude Opus 4.7 + Codex.

License

Promotion Decision

Attach run IDs, datasets, metrics, and reproduction commands.

Source Anchor

lume-commerce/README.md

Detected Structure

Method · Evaluation · References · Code Anchors · Architecture