Body As Input Overview

Body As Input Overview

The body-input layer gathers evidence about what the performer is doing. It must
work when only a camera is available, and it should improve when mocopi, watch,
phone, or depth sensors are available.

Current Input Classes

The architectural rule is camera-first: basic gestures must not stop working
because mocopi is off, charging, or disconnected.

Runtime Principle

Use sensors as evidence, not as a single point of failure.

camera sees left hand high
  -> eligible gesture
  -> mocopi/watch/depth can raise confidence
  -> safety gate debounces and dispatches intent

For DJ/AirDeck, this matters more than model sophistication. The user must be
able to raise a hand and get a predictable command even without wearing sensors.

MotionMixApp Input Path

MotionMixApp gathers:

• `CMDeviceMotion` into `SensorFrameFFI`;
• camera pose/statistics where enabled;
• mocopi features through `MocopiFeatureExtractor`;
• watch/pocket features;
• SAN capture frames through `SANTrajectoryLogger`.

`EchelonBridge` coordinates those inputs and feeds Rust Echelon plus downstream
services.

Mac4 / K11 Input Path

Mac4:

• receives Sony mocopi from the phone/sensors;
• feeds Unity on UDP `:9702`;
• relays low-rate mocopi state to MotionMix when stable.

K11:

• owns the camera/pose viewer lane;
• owns Rekordbox command safety;
• must preserve a camera-only gesture path.

Documents In This Section

• [imu-and-motion.md](imu-and-motion.md): CoreMotion and phone/wrist motion.
• [vision-pose.md](vision-pose.md): camera pose baseline.
• [mocopi-bones.md](mocopi-bones.md): mocopi feature extraction and caveats.
• [femto-bolt-depth.md](femto-bolt-depth.md): K11 camera/depth status and limits.
• [sensor-fusion.md](sensor-fusion.md): BodyTruth and evidence fusion policy.

Correct Boundary

Do not say:

- mocopi is required for hand raise;
- DELL is always the body-input engine;
- SAN was trained to handle every sensor-dropout case unless a training report
proves it;
- the 128D vector is one universal unchanged contract across every consumer.

Do say:

• camera pose is the baseline for visible-body gestures;
• optional sensors improve confidence and body richness;
• BodyTruth should become the shared authority after it is stable;
• source files decide what is currently implemented.