Mega serials start CL85x...

And also give me a script for both of them on how I should go about doing that for both of the reels on how I'm explaining the process, since we didn't have a live director, we'll just say# Femto Mega vs Femto Bolt — Differentiation + MediaPipe Setup Runbook

Last verified state: 2026-05-10 21:00Z (pairing plan), tasks #234-#247 closed.

---

Why this doc exists

Mac4 carries the dev rig with Femto Mega. K11 carries the bar product install with Femto Bolt. Both run the same pipeline: RGB feed in → MediaPipe BlazePose → 33 landmarks → echelon-bar Rust → LUMA UDP :9703 → Unity visuals. The cameras differ in physical form factor and lens characteristics, not in software path. This doc nails down what makes each one its own thing and how MediaPipe gets wired on K11.

---

The two cameras side-by-side

One-line difference: Mega has a brain on board and can talk Ethernet. Bolt is dumber and smaller. For our pipeline neither matters — we only use the RGB feed and run MediaPipe on the host.

---

What both cameras give us that we use

• RGB feed at 1280×720 @ 30fps (we don't need 4K — MediaPipe scales internally)
• Aligned depth (optional, currently disabled in `mediapipe_pose_verify.py`)
• IMU (currently unused; reserved for future tilt-aware effects)

What we don't use from either:
- Onboard body tracking (Orbbec public SDK has no body tracker on macOS arm64 or Windows arm64; works on Linux/Win x86_64 but quality lags MediaPipe in our tests)
- Multi-camera sync (single camera per machine)
- Audio mic array (UMA-8 handles audio on K11)

---

The pipeline (identical on Mac4 and K11)

Femto camera (RGB)
    │ pyorbbecsdk Python binding
    ▼
mediapipe_pose_verify.py      ← BlazePose GHUM 3D, 33 landmarks, 30fps
    │ JSONL per frame (timestamp, landmark x/y/z, world coords, visibility)
    ▼
echelon-bar (Rust binary)
    │ encoder maps 33 landmarks → 128D LatentState
    │ MotionSynth turns dynamics into audio events
    ▼
LUMA UDP datagrams on :9703 (local loopback)
    │
    ▼
Unity LumaUdpReceiver        ← bar product visuals

Every box exists on both machines. Same Rust crate (`crates/femto-bridge`), same Python verify harness, same Unity receiver code. Only the camera model (Mega vs Bolt) and OS-specific bits (CoreAudio vs WASAPI, launchd vs NSSM) differ.

---

Mac4 state (Femto Mega)

Verified 2026-05-09 13:08Z, task #234 ✅ PASS:

• Python venv at `[home-path]`, mediapipe 0.10.35 installed
• pyorbbecsdk source-built `.so` at `[home-path]`
• Verify script: `tools/femto-verify/mediapipe_pose_verify.py`
• Acceptance gate hit: 28.0 fps avg / 98.6
• Run command (in ttys004 because Femto is held by root publisher):

cd [home-path] && \
sudo -E DYLD_LIBRARY_PATH=$HOME/lume-femto/pyorbbecsdk/install/lib \
  PYTHONPATH=$HOME/lume-femto/pyorbbecsdk/install/lib \
  ./venv/bin/python tools/mediapipe_pose_verify.py --duration 15 \
  --out [home-path] +%Y%m%dT%H%M%S).jsonl

---

K11 state (Femto Bolt)

K11 is Windows 11 Pro, NOT Linux. AMD Ryzen 9 8945HS, 28.8 GB RAM, 898 GB free. User profile is `C:\Users\Mohamed Diomande` (space in path — always quote).

Verified 2026-05-10 21:00Z + per tasks #240-#247 ✅:

• Python 3.12.8 installed
• pyorbbecsdk installed (task #166 ✅)
• mediapipe + pose_landmarker model installed (task #241 ✅)
• Rust toolchain installed for `x86_64-pc-windows-gnu` host (task #240 ✅)
• Comp-Core cloned, on `feat/femto-only-bar` branch (task #242 ✅)
• Cargo build + tests green (task #243 ✅)
• Release build green (task #246 ✅)
• Live Femto Bolt verify passed via `run-on-k11.ps1` (task #244 ✅)
• 3 LUME NSSM services running: LUME-Depth :9700 / LUME-Audio :9701 / LUME-Mocopi :9702
• Tailscale [ip] (nucbox-k11)
• Femto Bolt USB-attached: Depth + RGB camera nodes both OK

---

MediaPipe setup on K11 (idempotent, do not re-run if already green)

This is the canonical setup procedure. If task #241 is already ✅ (it is), this is for reference / rebuild after a wipe.

1. Verify Python + pip

python --version    # expect 3.12.8
pip --version

If Python 3.12 isn't there, install from python.org. Don't use the Microsoft Store version — it sandboxes paths weirdly and breaks `pyorbbecsdk`.

2. Create the venv (avoid system Python pollution)

cd "C:\Users\Mohamed Diomande\lume-femto"
python -m venv venv
.\venv\Scripts\Activate.ps1
python -m pip install --upgrade pip

If PowerShell blocks the activate script, run once per user:

Set-ExecutionPolicy -Scope CurrentUser -ExecutionPolicy RemoteSigned

3. Install MediaPipe + numpy + opencv

pip install mediapipe==0.10.35 numpy opencv-python

`opencv-python` is needed for color conversion in the verify script. `mediapipe==0.10.35` is the version that matches Mac4 — pin it to avoid drift.

4. Download the BlazePose GHUM 3D model

The model file is `pose_landmarker_full.task` (or `_heavy.task` for higher accuracy). MediaPipe's pip package usually bundles a smaller default but for our acceptance gates we need GHUM 3D explicitly:

$model = "https://storage.googleapis.com/mediapipe-models/pose_landmarker/pose_landmarker_full/float16/1/pose_landmarker_full.task"
$dest = "C:\Users\Mohamed Diomande\lume-femto\models\pose_landmarker_full.task"
New-Item -ItemType Directory -Force -Path (Split-Path $dest)
Invoke-WebRequest -Uri $model -OutFile $dest

5. Install pyorbbecsdk (the Femto Bolt Python binding)

The Orbbec public SDK doesn't ship a Windows pip wheel. Build from source:

git clone https://github.com/orbbec/pyorbbecsdk.git "C:\Users\Mohamed Diomande\lume-femto\pyorbbecsdk"
cd "C:\Users\Mohamed Diomande\lume-femto\pyorbbecsdk"
mkdir build
cd build
cmake .. -DPYTHON_EXECUTABLE="C:\Users\Mohamed Diomande\lume-femto\venv\Scripts\python.exe"
cmake --build . --config Release

Resulting `.pyd` lands in `pyorbbecsdk\install\lib\`. Add that path to `PYTHONPATH` when running the verify script:

$env:PYTHONPATH = "C:\Users\Mohamed Diomande\lume-femto\pyorbbecsdk\install\lib"

6. Verify Femto Bolt enumeration

Plug Femto Bolt in. Check USB:

Get-PnpDevice -PresentOnly | Where-Object {$_.FriendlyName -match "Orbbec|Femto"} | Format-Table FriendlyName, Status

Expect at least two entries (RGB + Depth) with status `OK`.

7. Run the verify harness

cd "C:\Users\Mohamed Diomande\Desktop\Comp-Core"
git checkout feat/femto-only-bar
git pull origin feat/femto-only-bar
.\tools\femto-verify\run-on-k11.ps1

`run-on-k11.ps1` is the service-aware wrapper — it knows about LUME-Depth / LUME-Audio / LUME-Mocopi NSSM services and stays out of their way. The fixed `mediapipe_pose_verify.py` (task #245 ✅, depth-frame uint16 reshape) is what actually runs.

Acceptance gate: same as Mac4 — `avg fps ≥ 25, pose coverage ≥ 80

JSONL artifact lands in `C:\Users\Mohamed Diomande\lume-femto\proofs\femto-pose-<timestamp>.jsonl`.

8. Cargo build of echelon-bar

cd "C:\Users\Mohamed Diomande\Desktop\Comp-Core"
cargo build --release -p echelon-bar --target x86_64-pc-windows-gnu

If the GNU host gives MinGW ldscript trouble, fall back to `x86_64-pc-windows-msvc` with Visual Studio Build Tools installed.

---

How to differentiate them in code

Right now the verify script reads from "the first Orbbec device" — it doesn't care if it's Mega or Bolt. If we ever need to branch behavior per camera model, the device pipeline exposes a serial-prefix check:

device = pipeline.get_device()
info = device.get_device_info()
# Mega serials start CL85x...
# Bolt serials start CL83x...
if info.get_serial_number().startswith("CL83"):
    # Bolt-specific: shorter range, expect closer body
    pass

In practice we lock the room geometry per install, so the depth range delta doesn't matter and we don't branch.

---

Common gotchas

1. K11 user dir has a space. Always quote paths in PowerShell: `"C:\Users\Mohamed Diomande\..."`. Unquoted paths silently truncate at the space.

2. Femto USB-C cables matter. Use the cable that shipped with the camera. Generic USB-C cables drop frames at 30fps RGB. Symptom: `pose coverage` falls below 80

3. macOS arm64 has no body tracker SDK. Don't waste time looking for one. MediaPipe is the answer.

4. K11 NSSM services already bind :9700, :9701, :9702. Don't reuse those ports. echelon-bar uses :9703 for LUMA, that's locked.

5. NEVER touch `C:\lume\bin\*` (live publisher scripts) or the three NSSM services (`LUME-Depth`, `LUME-Audio`, `LUME-Mocopi`). They run production today.

6. GUI launch on K11 needs Session 1. SSH lands you in Session 0 — Unity Editor won't render there. Use `schtasks` to launch into Session 1, or RDP in and launch by hand.

7. Mediapipe 0.10.35 only. Newer versions changed the landmark callback signature; older versions don't have the GHUM 3D model wired by default. Pin it.

---

Where the files live

---

Acceptance summary

If you want to know "is this thing actually working," run one of these two from the Mac4 or K11 terminal:

Mac4 (Femto Mega): verify command in section above. Expect 28.0 fps / 98.6

K11 (Femto Bolt): `.\tools\femto-verify\run-on-k11.ps1`. Expect ≥ 25 fps / ≥ 80

If those pass, the camera + MediaPipe path is alive. Everything downstream (echelon-bar, LUMA, Unity) compounds on that.

---

What's next once both cameras are verified

1. Mac4 → K11 capture-corpus rsync. Phase E in the pairing plan.
2. K11 echelon-bar NSSM service. Wraps pose-pub + echelon-bar in one service that survives reboot (Phase D).
3. 1920×440 bar display. Plug it into K11, fullscreen Unity into it (tasks #178 + #211, hardware-gated).
4. Mocopi pairing. Add skeleton path alongside RGB pose for richer dynamics (task #170, sensor-gated).