Data Capture - Current Verified Paths

Data Capture - Current Verified Paths

This page documents what the current code can record. It does not claim that a
specific training run has already consumed the data unless a run artifact is
identified.

SAN JSONL Capture

Source: `MotionMixApp/Services/SANTrajectoryLogger.swift`.

Local output:

Documents/san-training/<session-id>.jsonl

Remote endpoint:

http://[ip]:9471/san-frame

Headers:

Content-Type: application/x-ndjson
X-MotionMix-SAN-Session: <session-id>
X-MotionMix-SAN-Schema: 2
X-MotionMix-SAN-Dims: 128

Subsampling:

subsampleRate = 6

At a 30Hz SAN step rate this targets roughly 5Hz capture.

Frame Contents

Each frame contains:

• schema version;
• dimensionality;
• timestamp;
• frame number;
• optional bar number;
• optional track name;
• optional track time;
• 128D input;
• four audio parameter groups;
• seven camera scores;
• cut timing;
• pattern intensity and variation;
• phrase lifecycle;
• gesture probabilities;
• phase;
• regime;
• calibration confidence;
• latency.

What This Captures

The logger captures SAN's flat input and SAN output at runtime. It is useful for:

• replay analysis;
• checking whether SAN output is flat or moving;
• future supervised training;
• comparing heuristic vs SAN-blended outputs;
• correlating track metadata with body state.

What This Does Not Capture By Itself

By itself, SAN JSONL does not prove:

• the current SAN weights were trained from this session;
• the session was accepted into a training set;
• a train/validation split exists;
• the model improved;
• Rekordbox gestures worked;
• mocopi, watch, and camera were all active.

Those require separate metadata and training reports.

Required Session Folder Structure Going Forward

Training sessions should be stored as:

session-YYYYMMDD-HHMMSS-<short-label>/
  manifest.json
  san/
    frames.jsonl
  video/
    raw-camera.<ext>
    preview.<ext>
  pose/
    body-truth.jsonl
    mocopi.jsonl
    camera-pose.jsonl
  audio/
    track-events.jsonl
    rekordbox-events.jsonl
  annotations/
    gestures.jsonl
    notes.md
  reports/
    capture-summary.md

`manifest.json` should include:

{
  "schema_version": 1,
  "session_id": "",
  "started_at": "",
  "device": "",
  "sources": {
    "camera": false,
    "mocopi": false,
    "watch": false,
    "sensor_logger": false,
    "rekordbox": false
  },
  "runtime_versions": {},
  "notes": ""
}

Training Provenance Rule

A future training page must cite:

• source session folders;
• accepted/rejected frame counts;
• script path;
• output checkpoint path;
• exported runtime artifact path;
• validation report path.

Do not put dataset counts into docs unless they came from that report.

K11 Bundle Reconstruction Returns

K11 now has a verified offline reconstruction return lane for processed
rehearsal bundles.

Canonical storage:

C:\lume\dance-sessions\_processed\<bundle_id>

Mac5-derived return files:

derived\sam3d\summary.json
derived\sam3d\sam3d_frames.jsonl
derived\sam3d\motion_windows.jsonl
derived\sam3d\template_candidates.json
derived\sam3d\worker.log
derived\sam3d\worker_result.json

The return lane is for training and review evidence. It is not a live control
lane. `summary.json` must state whether SAM3DBody actually produced a successful
frame. Placeholder statuses are valid only when explicit, for example
`skipped_no_usable_source_video`.

Verified audit on 2026-05-31:

bundle_count: 6
pass_count: 6
fail_count: 0

K11 SAM3D Review Index

After Mac5 returns reconstruction artifacts, K11 also receives a global review
index:

C:\lume\dance-sessions\sam3d-review-index.json
C:\lume\dance-sessions\sam3d-review-index.md
C:\lume\dance-sessions\sam3d-gesture-candidate-library.json
C:\lume\dance-sessions\sam3d-mapped-gesture-library.json
C:\lume\dance-sessions\sam3d-approved-gesture-library.json
C:\lume\dance-sessions\sam3d-dry-run-promotion-manifest.json
C:\lume\dance-sessions\sam3d-capture-targets.json
C:\lume\dance-sessions\sam3d-capture-session-plan.json
C:\lume\dance-sessions\sam3d-capture-session-plan.md
C:\lume\dance-sessions\sam3d-recording-queue.json
C:\lume\dance-sessions\sam3d-gesture-training-library.json
C:\lume\dance-sessions\sam3d-promotion-readiness.json
C:\lume\dance-sessions\sam3d-approval-status.json
C:\lume\dance-sessions\sam3d-review-decisions-template.json

The index schema is:

lume.sam3d_review_index.v1

It summarizes each processed bundle, its reconstruction status, whether it is
ready for human review, and any template candidates. It is deliberately
separate from live control. A candidate in this file is evidence for review,
not permission to send Rekordbox commands.

Verified 2026-05-31 summary:

bundle_count: 6
ready_for_human_review_count: 5
needs_attention_count: 0
live_enabled_candidate_count: 0

Verified 2026-06-01 mapping summary:

candidate_count: 13
mapped_exact_count: 0
review_only_count: 13
movement_target_count: 23
capture_target_count: 23
needs_capture_count: 21
live_baseline_needs_sam3d_count: 2
capture_session_count: 23
total_positive_takes: 69
total_negative_takes: 46
recording_queue_item_count: 115
gesture_training_library_count: 23
gesture_training_left_deck_count: 11
gesture_training_right_deck_count: 11
gesture_training_center_count: 1
gesture_training_scratch_or_rewind_count: 8
promotion_readiness_status_counts: needs_labeled_capture 21, needs_sam3d_reconstruction_for_live_camera_baseline 2
promotion_readiness_zero_live_counter_check: true
estimated_capture_minutes: 15.3
negative_movement_target_count: 27
attached_negative_control_count: 92
approved_count: 0
dry_run_item_count: 0
live_enabled_candidate_count: 0

`tier0-recorder-push-validation` now completes through the Mac5 label-window
fallback path. The fallback is recorded in `summary.json` so training code can
distinguish labeled gesture windows from regular sampled reconstruction
evidence.

AirDeck review UI:

C:\lume\airdeck-ui\sam3d-review.html
C:\lume\airdeck-ui\sam3d-mapped-gesture-library.json
C:\lume\airdeck-ui\sam3d-approved-gesture-library.json
C:\lume\airdeck-ui\sam3d-dry-run-promotion-manifest.json
C:\lume\airdeck-ui\sam3d-capture-targets.json
C:\lume\airdeck-ui\sam3d-capture-session-plan.json
C:\lume\airdeck-ui\sam3d-capture-session-plan.md
C:\lume\airdeck-ui\sam3d-capture-session-plan.html
C:\lume\airdeck-ui\sam3d-recording-queue.json
C:\lume\airdeck-ui\sam3d-recording-queue.html
C:\lume\airdeck-ui\sam3d-gesture-training-library.json
C:\lume\airdeck-ui\sam3d-gesture-training-library.html
C:\lume\airdeck-ui\sam3d-promotion-readiness.json
C:\lume\airdeck-ui\sam3d-promotion-readiness.html
C:\lume\airdeck-ui\sam3d-approval-status.json
C:\lume\airdeck-ui\sam3d-approval-console.html
C:\lume\airdeck-ui\sam3d-review-decisions-template.json

The AirDeck review page shows each bundle, candidate gesture, confidence,
fallback note, motion window, and evidence frame row. It links from the existing
AirDeck stage and movement-library pages. The companion candidate library uses
schema `lume.airdeck.sam3d_gesture_candidate_library.v1` and is non-live:
`approved_count = 0`, `live_enabled_candidate_count = 0`.

The K11 operator pages now runtime-load their sibling JSON artifacts when served
over HTTP, with the embedded generation snapshot as a fallback for local file
opens. `sam3d-review.html` fetches `sam3d-review-index.json`,
`sam3d-gesture-candidate-library.json`, `sam3d-mapped-gesture-library.json`,
`sam3d-approved-gesture-library.json`,
`sam3d-dry-run-promotion-manifest.json`, and `sam3d-capture-targets.json`.
The capture plan, recording queue, gesture training library, and approval
console similarly fetch their own JSON files and show whether they are using
sibling JSON or embedded fallback data. This makes the UI an actual JSON-backed
review surface instead of only a one-time static render.

The page can also export browser-local human decisions as
`lume.airdeck.sam3d_review_decisions.v1`. That decision export is still not a
live control artifact; a separate promotion manifest and dry-run proof remain
required before any Rekordbox command can be enabled.

The exported decision file can be fed back to the builder with
`--decisions <sam3d-review-decisions.json>`. Approved candidates are written to
`lume.airdeck.sam3d_approved_gesture_library.v1`, a separate non-live library.
If an approved gesture has an exact or selected AirDeck target, the dry-run
manifest can include it with `send_keys = false`. If the approval is visual-only
or lacks a selected target, it remains approved but blocked from dry-run command
simulation.

The approval console is the non-live operator status page for this handoff. It
reads `sam3d-approved-gesture-library.json`,
`sam3d-dry-run-promotion-manifest.json`, and
`sam3d-approval-status.json`, then shows the gates from exported review
decision to approved library to dry-run manifest. Current state is intentionally
empty: `decision_count = 0`, `approved_count = 0`, `dry_run_item_count = 0`,
and `live_enabled_candidate_count = 0`.

The approval path now has a local smoke test:

viz/lume-pcloud/tools/lume-mac5-reconstruction/smoke_test_sam3d_approval_path.py

It builds the current review artifacts into a temporary directory, creates one
temporary approved review decision, rebuilds with `--decisions --no-upload`,
and verifies that the decision lands only in
`sam3d-approved-gesture-library.json` / non-live dry-run artifacts. Current
verified smoke result approved the review-only `keep_signature` candidate into
the separate approved library, blocked it from dry-run because it has no AirDeck
target, and kept `send_keys = false`, `rekordbox_live_enabled = false`, and
`live_enabled_candidate_count = 0`.

The same smoke test also covers the future exact-target branch using the current
AirDeck movement target catalog. It synthesizes an approved
`left_hand_raise` candidate that exactly maps to the Deck 1 Play/Pause target
(`d1_play_pause`, key `z`). The result is one dry-run item, zero blocked
approved candidates, and still `send_keys = false`, `rekordbox_live_enabled =
false`, and `live_enabled_candidate_count = 0`. This proves that a future
SAM3D-backed left/right deck gesture can enter the dry-run manifest without
becoming live Rekordbox control.

The builder can now ingest exported review decisions directly from K11 without
manual local copying:

python3 tools/lume-mac5-reconstruction/build_k11_sam3d_review_index.py \
  --remote-decisions 'C:\lume\airdeck-ui\sam3d-review-decisions.json'

That intake path now has its own one-shot runner and status page:

cd [home]/Desktop/lume-commerce
python3 viz/lume-pcloud/tools/lume-mac5-reconstruction/run_k11_sam3d_approval_intake.py
python3 viz/lume-pcloud/tools/lume-mac5-reconstruction/run_k11_sam3d_approval_intake.py --run

The default command is status-only. It checks for:

C:\lume\airdeck-ui\sam3d-review-decisions.json

and writes:

C:\lume\airdeck-ui\sam3d-approval-intake-status.json
C:\lume\airdeck-ui\sam3d-approval-intake-status.html
C:\lume\dance-sessions\sam3d-approval-intake-status.json

With `--run`, if the decision file exists and has schema
`lume.airdeck.sam3d_review_decisions.v1`, the runner rebuilds the review
artifacts through `--remote-decisions`, then runs the full local+K11 verifier.
The official current state is `waiting_for_decisions`, because no exported
human decision file exists on K11 yet. That is the correct safe state:
`decision_count = 0`, `approved_count = 0`, `dry_run_item_count = 0`, and
`live_enabled_candidate_count = 0`.

The companion `sam3d-review-decisions-template.json` lists the current 13
candidate IDs and their target options. A smoke test using a temporary K11
decision file approved one review-only candidate, wrote it into the separate
approved library, blocked it from dry-run because it had no AirDeck target, and
kept `live_enabled_candidate_count = 0`.

The mapped gesture library joins each SAM3D candidate to the AirDeck movement
library by exact label only. Current SAM3D candidates are visual/review labels
(`keep_signature`, `wave_color`, baseline/recovery, unlabeled/source-missing),
so they remain review-only. The mapping file still carries the current AirDeck
target catalog for left/right deck control, transport, next-track, sync, loop,
scratch nudge, and platter-spin rewind/forward gestures. The dry-run promotion
manifest is empty until a browser-local decision export approves a candidate;
even then it sets `send_keys = false` and `rekordbox_live_enabled = false`.

The capture target file is the forward work queue for the AirDeck library. It
is generated from K11 `movement-library.json` and groups 23 positive targets:
11 left-deck gestures, 11 right-deck gestures, and one center safe-stop. It
includes transport, sync, next track, loop, scratch nudge, and platter-spin
forward/rewind targets. The two hand-raise targets already exist as live camera
baselines but still need SAM3D reconstruction evidence; the remaining 21
targets need labeled K11 capture bundles before review/promotion.

The capture-session plan is the operator-facing form of that queue. It expands
the 23 targets into 23 non-live rehearsal sessions with 69 positive takes and
46 negative takes, about 15.3 active minutes of capture. The HTML plan is linked
from `sam3d-review.html` and does not replace the older AirDeck
`capture-plan.html`; it is specifically the SAM3D evidence plan for left/right
deck control, scratch/rewind zones, loop/next-track actions, hand raises, and
the center safe-stop. It never sends Rekordbox keys and keeps
`live_enabled_candidate_count = 0`.

The recording queue is the take-by-take form of the same plan. It expands the
23 sessions into 115 explicit rows: 69 positive gesture takes and 46 negative
or no-op control takes. Every row carries a stable label stub, prompt, gesture
window, expected command/key for positive takes, expected no-op state for
negative takes, and the same safety flags: `send_keys = false` and
`live_enabled = false`. `sam3d-review.html` and the capture-session plan both
link to `sam3d-recording-queue.html`, so the operator can move from review to
capture planning to recordable rows without touching the live Rekordbox gate.

The gesture training library is the consolidated, non-live vocabulary built
from those capture targets and rows. It writes
`lume.airdeck.sam3d_gesture_training_library.v1` with 23 gesture specs: 11 left
deck, 11 right deck, and one center safe-stop. It makes the control semantics
explicit for training and review: active hand, hand policy, deck surface, zone
kind, motion role, motion direction, command intent, expected command/key,
positive cues, veto cues, no-op controls, queue take IDs, and capture status.
Eight specs are scratch/rewind or scratch/nudge saucer-zone commands. The
training-library page is linked from the review, capture, recording, and
approval pages and still keeps `send_keys = false`, `rekordbox_live_enabled =
false`, and `live_enabled_candidate_count = 0`.

The promotion-readiness audit is the machine-checkable gate between training
evidence and any later approval/dry-run path. It writes
`lume.airdeck.sam3d_promotion_readiness.v1`, one row per gesture spec, with the
current blocker for each command. Current state is intentionally not promoted:
21 gesture specs need labeled K11 capture, two live camera hand-raise baselines
need SAM3D reconstruction evidence, zero gestures are approved, zero are
dry-run-ready, and every artifact live counter is zero. The audit also records
the global gates: JSON artifacts exist, human decisions are not yet imported,
approved library is empty, dry-run manifest is empty, all live counters are
zero, and this builder does not create a live promotion manifest.

The capture targets now also carry annotation metadata for training: deck
surface, active hand, hand policy, zone kind, motion role, command intent,
motion direction, positive cues, veto cues, and annotation tags such as
`deck:deck_1_left`, `hand:left`, `role:scratch_or_rewind_backward`, and
`intent:rewind_or_scratch`. The builder imports the no-op safety rows from
K11 `movement-library.json` as 27 negative movement targets and attaches 92
specific no-op controls across the 23 capture sessions. Those controls are
labels for training and review only; they still have no command, no key send,
and no live promotion.

First SAM3D Capture Batch

The first concrete batch is now cut from the 115-row recording queue. It is not
another dashboard; it is a small operator batch for the first real capture pass:

schema: lume.airdeck.sam3d_first_capture_batch.v1
batch_id: sam3d-first-left_hand_raise-d5fdae4bee6de116
label: left_hand_raise
items: 5
positive: 3
negative/no-op: 2
estimated active capture: 40 seconds
send_keys: false
rekordbox_live_enabled: false

K11 paths:

C:\lume\airdeck-ui\sam3d-first-capture-batch.json
C:\lume\airdeck-ui\sam3d-first-capture-batch.md
C:\lume\airdeck-ui\sam3d-first-capture-batch.html
C:\lume\airdeck-ui\sam3d-first-capture-status.html
C:\lume\airdeck-ui\sam3d-first-capture-pipeline-report.json
C:\lume\dance-sessions\sam3d-first-capture-batch.json
C:\lume\dance-sessions\sam3d-first-capture-batch.md
C:\lume\dance-sessions\sam3d-first-capture-pipeline-report.json
C:\temp\capture_sam3d_first_batch.cmd
C:\temp\package_k11_sam3d_first_capture_bundles.py

The AirDeck UI now includes `sam3d-first-capture-batch.html`, a browser
operator page for the five-take first batch. It shows the positive and negative
Pose Coach commands, expected bundle IDs, post-recording Mac1 status/watch
commands, and the safety invariant. It does not execute commands from the
browser and does not send Rekordbox keys. The main SAM3D review page links to
this first-batch page.

The Mac1 pipeline runner now also publishes
`sam3d-first-capture-pipeline-report.json` and
`sam3d-first-capture-status.html` into the K11 AirDeck UI. That status page
shows the current package/reconstruction state, the missing or packaged take
rows, expected bundle IDs, and next action. Current verified state is:

status: waiting_for_recordings
expected_take_count: 5
missing_count: 5
packaged_count: 0
live_enabled_candidate_count: 0

The runbook uses camera-only Pose Coach evidence commands by default:

C:\temp\record_airdeck_evidence.cmd "left_hand_raise" 8 upper_body_hands_face
C:\temp\record_airdeck_negative.cmd "negative_scratch_body" 8 upper_body_hands_face
C:\temp\record_airdeck_negative.cmd "negative_loop_hover" 8 upper_body_hands_face

That is intentional. Mocopi can enrich the capture, but it must not block this
first evidence batch. Guarded promotion-candidate capture remains a later pass
after SAM3D evidence and human approval.

After the five takes are recorded, K11 now has a dedicated packager for the
missing handoff into Mac5 reconstruction:

python C:\temp\package_k11_sam3d_first_capture_bundles.py ^
  --batch C:\lume\airdeck-ui\sam3d-first-capture-batch.json ^
  --root C:\lume\dance-sessions ^
  --pose-root C:\lume\recordings\pose-coach\sessions

The packager creates one `lume.rehearsal_bundle.v1` bundle per take under
`C:\lume\dance-sessions\_processed`, copies the Pose Coach raw-camera AVI and
training JSONL into the bundle, writes a SAM3D label-window file, updates
`bundle-index.json`, and writes
`C:\lume\dance-sessions\sam3d-first-capture-bundle-report.json`. A dry-run on
2026-06-01 correctly found zero matching sessions because the five first-batch
takes had not yet been recorded; it did not create fake bundles.

Expected bundle IDs are deterministic and already written into the batch JSON:

sam3d-first-left_hand_raise-d5fdae4bee6de116-take_43245a66e7fec4ea
sam3d-first-left_hand_raise-d5fdae4bee6de116-take_e990d0ffd66ac655
sam3d-first-left_hand_raise-d5fdae4bee6de116-take_8f1801a1dabc2076
sam3d-first-left_hand_raise-d5fdae4bee6de116-take_e1499ca7e3330981
sam3d-first-left_hand_raise-d5fdae4bee6de116-take_27ef7722c3d0dcf6

Once those bundles exist, Mac1 should route them to Mac5 with:

cd [home]/Desktop/lume-commerce
python3 viz/lume-pcloud/tools/lume-mac5-reconstruction/orchestrate_k11_to_mac5.py \
  --bundle sam3d-first-left_hand_raise-d5fdae4bee6de116-take_43245a66e7fec4ea \
  --bundle sam3d-first-left_hand_raise-d5fdae4bee6de116-take_e990d0ffd66ac655 \
  --bundle sam3d-first-left_hand_raise-d5fdae4bee6de116-take_8f1801a1dabc2076 \
  --bundle sam3d-first-left_hand_raise-d5fdae4bee6de116-take_e1499ca7e3330981 \
  --bundle sam3d-first-left_hand_raise-d5fdae4bee6de116-take_27ef7722c3d0dcf6 \
  --max-frames 3 --clean --run-id sam3d-first-capture

There is also a one-shot Mac1 runner that handles the entire post-recording
handoff. Status-only mode is safe and does not mutate K11; it currently reports
`waiting_for_recordings` until the five Pose Coach takes exist:

cd [home]/Desktop/lume-commerce
python3 viz/lume-pcloud/tools/lume-mac5-reconstruction/run_k11_sam3d_first_capture_pipeline.py

After recording, run:

cd [home]/Desktop/lume-commerce
python3 viz/lume-pcloud/tools/lume-mac5-reconstruction/run_k11_sam3d_first_capture_pipeline.py --run

If Mac1 should wait while K11 is recording, use the watcher:

cd [home]/Desktop/lume-commerce
python3 viz/lume-pcloud/tools/lume-mac5-reconstruction/run_k11_sam3d_first_capture_pipeline.py --run --watch

The watcher polls K11 with the packager in dry-run mode. If the five takes are
still missing, it reports `waiting_for_recordings`; it does not package partial
data or fabricate evidence. Only after all expected recordings are detected
does it run the real packaging and Mac5 reconstruction stages.

That command packages K11 sessions, runs Mac5 reconstruction on the resulting
bundle IDs, syncs lightweight evidence assets back from Mac5 to K11, rebuilds
the review index, and verifies the AirDeck artifacts. It still does not create
a live promotion manifest and still stops before human approval. The asset sync
copies review JPG frames, CSV outputs, and SAM3D logs into each K11 bundle under:

C:\lume\dance-sessions\_processed\<bundle>\derived\sam3d\assets\

and writes:

C:\lume\dance-sessions\_processed\<bundle>\derived\sam3d\evidence_assets.json

`sam3d-review.html` now uses that manifest to render actual evidence-frame
thumbnails with CSV/log links. On 2026-06-01, seven existing evidence frames
were synced from Mac5 to K11 across the six current bundles. If the pipeline is
run manually instead, sync assets first:

python3 viz/lume-pcloud/tools/lume-mac5-reconstruction/sync_k11_sam3d_evidence_assets.py --bundle <bundle_id>

then rebuild the review index with:

python3 viz/lume-pcloud/tools/lume-mac5-reconstruction/build_k11_sam3d_review_index.py

The review verifier now treats evidence thumbnails as a hard gate, not a UI
assumption. `verify_k11_sam3d_airdeck_artifacts.py` checks:

- the review page contains the evidence-frame renderer;
- each successful evidence frame has a K11 review href;
- each synced frame reports `asset_sync_status = synced_to_k11`;
- CSV and log links exist when the source frame reports those assets;
- each bundle-level `evidence_assets.json` manifest has schema
`lume.sam3d_evidence_assets.v1`;
- manifest frame counts match the review index frame counts;
- K11 `Test-Path` confirms the referenced frame, CSV, and log files exist and
are non-empty.
- the first-capture batch JSON/HTML exist in the AirDeck UI and keep the same
zero-live safety flags.
- the first-capture pipeline status JSON/HTML exist in the AirDeck UI, report
a known pipeline state, and preserve the non-live/human-approval safety flags.
- the gesture training library is semantically complete, not just present:
23 gesture specs, 11 left deck, 11 right deck, 1 center mixer/safe-stop,
8 scratch/rewind/saucer specs, and zero live-enabled rows.
- left-deck specs require `active_hand = left`, right-deck specs require
`active_hand = right`, and center-mixer safe stop requires `active_hand = both`.
- positive queue rows require expected command/key metadata while negative/no-op
rows require empty commands, `expected_key = none`, and `send_keys = false`.
- the 115-row recording queue must cover every capture target and every capture
session exactly once through its positive and negative takes.

Verified 2026-06-01 local + K11 summary:

status: pass
bundle_count: 6
evidence_asset_manifests: 6
evidence_frames: 7
synced_evidence_frames: 7
first_capture_batch_items: 5
first_capture_pipeline_status: waiting_for_recordings
first_capture_missing: 5
approved_count: 0
dry_run_item_count: 0
live_enabled_candidate_count: 0
zero_live_counter_check: true

Capture Mesh Preflight

The operator UI now has a live-readiness snapshot before recording starts:

C:\lume\airdeck-ui\sam3d-capture-mesh-preflight.html
C:\lume\airdeck-ui\sam3d-capture-mesh-preflight.json
C:\lume\dance-sessions\sam3d-capture-mesh-preflight.json

Mac1 generates it with:

cd [home]/Desktop/lume-commerce
python3 viz/lume-pcloud/tools/lume-mac5-reconstruction/build_k11_sam3d_capture_mesh_preflight.py

The page checks the required K11 capture path and the optional Mac4/MotionMix
evidence path:

- K11 SSH reachability;
- K11 AirDeck UI root, dance-session root, first-capture batch JSON, and
`C:\temp\capture_sam3d_first_batch.cmd`;
- visible K11 camera devices;
- current first-capture pipeline status and missing-recording count;
- Mac4 mocopi LaunchAgent/sidecar process;
- Mac4 Sony UDP `:12351`, OSC `:9500`, and Unity `:9702` visibility;
- MotionMix `/health`, `/mocopi/health`, and `/body-truth` reachability.

The classification is conservative. K11 camera/runbook readiness is required.
Mac4 mocopi and MotionMix are optional evidence feeds, so the camera-only K11
recording path can still continue if Sony sensors are off or MotionMix is stale.
The preflight artifact itself is non-live and repeats the safety invariants:
`send_keys=false`, `rekordbox_live_enabled=false`, K11 remains the command gate,
and human approval plus a separate live-promotion manifest are still required.

Verified 2026-06-01:

capture_mesh_preflight_status: ready
k11_camera_count: 3
k11_required_ready: true
mac4_optional_ready: true
motionmix_health_ok: true
required_failures: 0
warnings: 0
live_enabled_candidate_count: 0

Mac4 is integrated as optional sensor evidence. Its mocopi sidecar is managed
by `com.lume.mocopi-sidecar-mac4`, listens on Sony UDP `:12351`, and now relays
to MotionMix through the Tailscale URL:

http://[ip]:9404/mocopi/state

MotionMix health was reachable at `http://[ip]:9404/health`, but
current BodyTruth was absent because no mocopi or camera body feed was active
at the time of the preflight. K11 camera devices were visible as Orbbec Femto
Bolt RGB and depth cameras, and Pose Coach was stopped until recording begins.

SAM3D Operator Console

The K11 UI now has one operator-facing status page that reads the same JSON
artifacts as the rest of the SAM3D/AirDeck handoff:

C:\lume\airdeck-ui\sam3d-operator-console.html
C:\lume\airdeck-ui\sam3d-operator-console.json
C:\lume\dance-sessions\sam3d-operator-console.json

Mac1 generates it as part of the review-index build:

cd [home]/Desktop/lume-commerce
python3 viz/lume-pcloud/tools/lume-mac5-reconstruction/build_k11_sam3d_review_index.py

The operator console is not a new control surface. It is a status aggregator
that merges:

- SAM3D review-index counts;
- the reviewable SAM3D candidate queue, including confidence, mapping status,
fallback notes, and the first synced evidence-frame/CSV/log links;
- the next recording rows from the 115-row recording queue;
- the left/right deck gesture-library summary, including scratch/rewind and
hand-specific coverage;
- capture-target and recording-queue counts;
- first-capture batch and first-capture pipeline status;
- capture mesh preflight status;
- approval-intake status;
- promotion readiness and zero-live counters.

Its current next action is derived from the artifacts, not hard-coded:

operator_next_action: record_first_k11_capture_batch
operator_zero_live: true

That means the system is ready for the first physical K11 recording batch, but
it is not ready for gesture promotion. The five expected first-capture takes do
not exist yet, no browser review decisions have been exported, and no dry-run or
live Rekordbox manifest has been created.

Verified 2026-06-01:

local_verifier_status: pass
remote_k11_verifier_status: pass
capture_mesh_preflight_status: ready
k11_camera_count: 3
mac4_optional_ready: true
approval_intake_status: waiting_for_decisions
first_capture_pipeline_status: waiting_for_recordings
first_capture_missing: 5
bundles: 6
ready_for_human_review: 5
evidence_frames: 7
synced_evidence_frames: 7
candidates: 13
operator_review_queue_count: 13
operator_recording_next_count: 16
targets: 23
sessions: 23
queue_items: 115
gestures: 23
gesture_summary_left_deck: 11
gesture_summary_right_deck: 11
gesture_summary_scratch_or_rewind: 8
gesture_summary_hand_specific: 22
approved: 0
dry_run: 0
live: 0
zero_live: true

Vast.ai or another rented GPU should be treated as an optional offline worker,
not a live command path. It can help if SAM3D reconstruction or training becomes
too slow on Mac5, but live AirDeck gestures should stay local on K11 because
WAN upload and round-trip latency are the wrong failure mode for Rekordbox
control.

The correct rented-GPU shape is:

K11 camera/Pose Coach capture
  -> K11 bundle packaging under C:\lume\dance-sessions\_processed
  -> optional upload of raw bundles to Vast.ai / workspace storage
  -> SAM3D reconstruction, feature extraction, or model training
  -> returned summaries, frames, motion windows, candidates, and logs
  -> K11 review UI and approved non-live gesture library
  -> dry-run manifest only after human approval
  -> separate local K11 live promotion gate

The expected speed profile is good for offline work and wrong for live control.
A 4090/A100 class rented GPU should be fast enough for batch SAM3D runs,
feature extraction, and overnight gesture-model experiments once data transfer
is accounted for. It is not the right place to decide whether a raised hand
should play/pause Rekordbox in the moment. Live decisions should be camera-local
on K11, with Mac4 mocopi and MotionMix as optional evidence feeds, and with the
rented GPU only improving the reviewed/trained library that K11 later consumes.