LUME Full System Goal

LUME Full System Goal

Created: `2026-06-10T05:36:17Z`
Status: `in_progress`
Mode: `post_mac4_green_wiring`

Goal

Build the complete LUME motion system from capture to command, without
collapsing the safety boundaries between machines. The system should accept
multi-angle body evidence from phones, iPads, Mac2, Mac4, and optional depth or
wearable sensors; package that evidence into one auditable session bundle; let
Mac5 reconstruct and learn offline; let Mac4 render, map, and prove read-only
output lanes; and let K11 remain the only machine allowed to send Rekordbox or
AirDeck commands.

The final proof is not a document. The final proof is one real captured session
where the bundle contains synchronized evidence, Mac5 produces reconstruction
artifacts from that evidence, SAN produces phrase/gesture/audio learning rows,
Mac4 produces read-only visual/audio mapping proof, and K11 is still the only
command authority.

Current State

Mac4-first is green. The current unlock artifact is:

`[home]/Desktop/MotionMix/lume-wiring/lume_mac4_first_completion_gate_20260610T035207Z.json`

That report has `completion_ready=true`,
`mac5_heavy_reconstruction_allowed=true`, wrapper status `pass`, audit status
`pass`, and no failed requirements. This unlocks Mac5 preparation and dry-runs,
but it does not by itself prove a real reconstruction. A real reconstruction
still requires fresh multi-angle body/camera evidence.

The post-Mac4 wiring package is already generated and synced to Mac2 and Mac5:

- Green unlock:
`[home]/Desktop/MotionMix/lume-wiring/lume_post_mac4_green_unlock_20260610T052000Z.json`
- Session bundle contract:
`[home]/Desktop/MotionMix/lume-wiring/lume_post_mac4_session_bundle_contract_v1.json`
- SAN/reconstruction boundary:
`[home]/Desktop/MotionMix/lume-wiring/lume_post_mac4_san_reconstruction_boundary_v1.json`
- Home-return checklist:
`[home]/Desktop/MotionMix/lume-wiring/lume-post-mac4-home-return-checklist-20260610T052000Z.md`
- Full-system plan:
`[home]/Desktop/MotionMix/lume-wiring/lume-post-mac4-full-system-plan-20260610T052000Z.md`
- Wiring report:
`[home]/Desktop/MotionMix/lume-wiring/lume_post_mac4_full_system_wiring_report_20260610T052000Z.json`

Architecture Boundary

K11 is the command plane. It runs Pose Coach, receives MotionMix pose packets,
tracks SAN/gesture state, gates promoted commands, and is the only place where
Rekordbox or AirDeck commands may be emitted. Phones, iPads, Mac2, Mac4, Mac5,
SAN exports, reconstruction workers, Unity, ENTHEA, MRT2, DEMON, and camera
tools are evidence or output lanes only.

Mac4 is the read-only stage and rendering plane. It runs BodyTruth, the
read-only output bridge, ENTHEA/Unity visuals, MRT2/Magenta-style audio mapping,
and DEMON/control-curve handoff. Mac4 can receive body/camera state, produce
visual or audio mapping proof, and write artifacts. Mac4 must not send keys,
notes, Rekordbox commands, or AirDeck commands.

Mac5 is the reconstruction and learning plane. It consumes session bundles,
runs dry-runs first, then runs heavy reconstruction only when real evidence is
present. Mac5 may produce meshes, pose reconstructions, view reliability
scores, template candidates, SAM3D/MAMMA-style derived artifacts, and learning
reports. Mac5 cannot command Rekordbox.

Mac2 is the external camera plane. Its job is to provide a wide room angle,
currently centered on the Insta360 when the camera is actually attached and
enumerated as an eligible external camera. FaceTime and screen capture stay
excluded because they are not reliable evidence sources for reconstruction.

Mac1 is the orchestration plane. It owns the audit scripts, generated wiring
artifacts, sync, documentation, validation, and operator-facing reports.

The iPhones are mobile pose/camera evidence planes. The iPhone 16 Plus,
iPhone 16 Pro Max, and iPhone 14 Pro Max should each publish labeled MotionMix
pose/camera evidence with device identity, timestamps, confidence, camera role,
and source id. They may influence K11 only through K11's bridge and promotion
gate. They do not command Rekordbox directly.

The iPads are operator memory and optional view planes. StageView and ShootView
can capture stage context, notes, stills, timing, and optional viewpoints. They
are useful, but the first reconstruction proof should not depend on both iPads
being online.

Arducam, Femto, Bolt, Mega, Insta360, and mocopi are sensor lanes. They are
valuable because they improve viewpoint diversity, depth, skeleton confidence,
or room coverage. They are not command surfaces.

Device Map

Data Lanes

The command lane is K11 only. Its artifacts are bridge mode proof, live/dry-run
status, command manifest, gesture promotion logs, and the latest pose/gesture
state. Its invariant is simple: if a command reaches Rekordbox, the chain must
show that it passed through K11.

The phone lane is MotionMix. Each phone must publish device identity, source id,
timestamps, pose landmarks, visible joint count, confidence, camera role, and
optionally video/still references. Multiple phones should be treated as
separate angles, not as one anonymous iPhone source.

The external camera lane is Mac2/Insta360. It must prove that an eligible
external camera exists before recording. The room-wide clip or reference frames
go into `raw/video` and the source manifest.

The Mac4 output lane is read-only. It includes BodyTruth state, ENTHEA/Unity
visual output, MRT2/Magenta-style audio mapping, DEMON control-curve handoff,
and camera/pose evidence routing. The MRT2 proof sink may receive CCs for proof,
but Mac4 still cannot command Rekordbox.

The SAN lane is phrase/audio/gesture learning. It captures dynamics, gesture
probabilities, phrase state, audio mapping features, camera score priors,
latency, and optional track metadata. SAN rows can annotate reconstruction
windows, but SAN cannot replace synchronized camera/pose evidence.

The reconstruction lane is Mac5. It consumes `raw/pose`, `raw/video`,
`raw/calibration`, source manifests, green unlock proof, and reconstruction
requests. It produces `derived/mac5`, `derived/sam3d`, `derived/reconstruction`,
view reliability scores, template candidates, and audit reports.

The semantic lane is N'Ko/MnKO. It can provide phrase labels, symbolic memory,
prompt language, cultural structure, and meaning tags for sessions. It is not a
camera source and is not required to unlock the first reconstruction. It becomes
more important after the capture/reconstruction path can prove body motion.

Session Bundle Contract

Every serious capture should resolve into one bundle root with this layout:

bundle.json
raw/sources/source_manifest.json
raw/calibration/camera_topology.json
raw/pose/{source_id}.jsonl
raw/video/{source_id}.{mp4|mov|avi}
raw/san/san_frames.jsonl
raw/operator/notes.json
derived/mac4_first/completion_gate.json
derived/san/training_index.json
derived/reconstruction/request.json
derived/mac5/dry_run_report.json
derived/mac5/reconstruction_report.json

The minimum first real capture requires at least three evidence sources:

1. K11 command/safety lane.
2. One phone pose/camera lane.
3. One external room-wide or Mac4 rich camera lane.

The recommended first real capture is stronger:

1. K11 safety/command lane.
2. iPhone 16 Plus MotionMix.
3. iPhone 14 Pro Max MotionMix.
4. Mac2 Insta360 room-wide.
5. Mac4 BodyTruth depth or wide evidence.
6. Optional iPhone 16 Pro Max, iPad StageView, iPad ShootView, and mocopi.

Every source needs `session_id`, `bundle_id`, `source_id`, `device_label`,
`device_model`, `machine`, `camera_role`, UTC timestamp, monotonic timestamp,
confidence, coordinate space, calibration status, and privacy/storage policy.

Execution Phases

Phase 0 is freeze. Preserve Mac4 green as the unlock state and stop treating
Mac4 as the blocker. The green report is now the baseline. Future work should
only reopen Mac4 if the proof goes stale or a required service breaks.

Phase 1 is bundle tooling. Build the validator and fixture generator around
the session bundle contract. This can be done while away from the devices. The
validator should accept a bundle path, verify required sources, verify green
unlock, verify K11-only command authority, verify SAN/reconstruction separation,
and produce a pass/fail report.

Phase 2 is dry-run Mac5. Create a synthetic or fixture bundle that satisfies
the file layout without pretending to be real capture evidence. Run Mac5
dry-run against that bundle and prove that Mac5 can parse the contract, route
inputs, write reports, and refuse heavy reconstruction when evidence is fake or
missing.

Phase 3 is home-return validation. When the devices are physically available,
run the checklist: K11 health, phone labels, dominant phone peer, MotionMix
freshness, Mac2 external camera enumeration, Mac4 BodyTruth/read-only bridge,
and camera framing.

Phase 4 is first real capture. Record one 20-60 second standing session with
clear body framing. If the session is upper-body-only, mark it that way and do
not call it full-body reconstruction proof. Package all sources into one bundle.

Phase 5 is Mac5 reconstruction. Run dry-run first, then heavy reconstruction
only after the validator confirms real evidence. Write the final report with
captured sources, reconstruction outputs, learned templates, failures, and next
calibration work.

Phase 6 is live promotion. Only after reconstruction and SAN evidence agree
should any gesture template be considered for K11 command promotion. Promotion
requires human review and K11 dry-run before live control.

Immediate Next Steps

While away from the devices, the right work is code readiness:

1. Build `verify_lume_post_mac4_session_bundle.py`.
2. Build a synthetic fixture bundle generator for Mac5 dry-run.
3. Wire the bundle contract into existing K11-to-Mac5 bundle packaging.
4. Add a report that separates `capture_ready`, `dry_run_ready`,
`reconstruction_ready`, and `live_promotion_ready`.
5. Keep syncing artifacts to Mac2 and Mac5, but do not start heavy
reconstruction from stale or placeholder data.

Current code-readiness update:

1. `verify_lume_post_mac4_session_bundle.py` exists and separates
`capture_ready`, `dry_run_ready`, `reconstruction_ready`, and
`live_promotion_ready`.
2. `build_lume_post_mac4_fixture_bundle.py` creates dry-run-only synthetic
fixtures that cannot be mistaken for real capture proof.
3. `run_lume_post_mac4_bundle_dry_run.py` writes
`derived/mac5/dry_run_report.json`.
4. `assemble_lume_post_mac4_real_capture_bundle.py` now turns a filled
source-plan JSON into a verified real-capture bundle.
5. The home-return source-plan template lives at
`lume-post-mac4-real-capture-source-plan-template.json`.

When back near the devices, the right work is physical validation:

1. Open MotionMix on the iPhones and confirm labels/fresh pose frames.
2. Confirm K11 bridge mode and command safety.
3. Confirm Mac2 sees the Insta360 as an eligible external camera.
4. Confirm Mac4 BodyTruth/output bridge are reachable and read-only.
5. Fill `lume-post-mac4-real-capture-source-plan-template.json` with the real
evidence paths.
6. Run `assemble_lume_post_mac4_real_capture_bundle.py` to package and verify
the first real bundle.
7. Run Mac5 dry-run, then real reconstruction only if the verifier reports
`reconstruction_ready=true`.

Done Criteria

The goal is complete only when all of these are true:

1. Mac4 green unlock is preserved and referenced inside the real session
bundle.
2. K11 is proven to be the only Rekordbox/AirDeck command gate.
3. At least one real multi-angle bundle exists with synchronized pose/camera
evidence.
4. SAN rows are exported and linked as annotations, not substituted for
reconstruction proof.
5. Mac5 dry-run passes against the bundle.
6. Mac5 heavy reconstruction runs against real evidence and writes a report.
7. The report names what was captured, what was reconstructed, what was learned,
what failed, and what should be calibrated next.
8. No phone, iPad, Mac2, Mac4, Mac5, Unity, ENTHEA, MRT2, DEMON, or
reconstruction process sends Rekordbox commands outside K11.

Risks

The largest risk is treating stale or partial evidence as if it were a real
capture. The validator must make that difficult. A stale pose packet, a
FaceTime camera, an upper-body-only frame, a missing source id, or an
unlabeled phone must produce an explicit degraded state.

The second risk is overloading K11. K11 should make decisions, not do all heavy
work. It can receive pose, gate commands, record SAN state, and show Pose
Coach. Reconstruction belongs on Mac5. Visual/audio rendering belongs on Mac4.

The third risk is confusing SAN with reconstruction. SAN is valuable because it
captures timing, gesture policy, phrase state, and audio response. It does not
prove camera geometry or body mesh quality.

The fourth risk is camera ambiguity. Three iPhones are useful only if they are
labeled and treated as separate angles. The system must not collapse them into
one generic `iphone_motionmix` source for reconstruction.

Operating Rule

Do not promote anything to live command until the chain is:

real capture -> validated bundle -> Mac5 reconstruction/dry-run report -> SAN
annotation -> human review -> K11 dry-run -> K11 live promotion.

K11 commands. Mac4 observes. Mac5 reconstructs. Everything else is evidence.