What Is Computational Choreography?

What Is Computational Choreography?

Computational choreography is the LUME/MotionMix idea that body motion is not
only something to record. It is a live control signal for sound, visuals, camera
direction, DJ commands, and eventually motion inscription.

The current system is not one finished neural model. It is a set of connected
runtime lanes:

- MotionMixApp on iPhone: Swift shell, Rust `cc-echelon` core, SAN service,
camera node, sensor logging, optional CoreML generation.
- Comp-Core: Rust crates for Echelon brain/audio/motion bridge, semantic
inscription, and N'Ko language infrastructure.
- Mac4/K11: Unity visuals, mocopi/Mega/Bolt/body feeds, MotionMix hub, AirDeck
Rekordbox safety bridge.
- Computational choreography docs: explanation layer. These docs must follow the
source, not invent names that the source does not support.

Verified Core Claim

The system converts body signals into compact state and uses that state to drive
generative behavior in real time.

Verified source anchors:

• `MotionMixApp/Services/EchelonBridge.swift`
• `MotionMixApp/Services/SANService.swift`
• `MotionMixApp/Services/SANTrajectoryLogger.swift`
• `MotionMixApp/Services/DiffusionService.swift`
• `MotionMixApp/Services/ClaimBridgeService.swift`
• `Comp-Core/core/audio-media/cc-echelon/crates/cc-brain/src/`
• `Comp-Core/core/semantic/cc-inscription/`
• `Comp-Core/core/semantic/cc-semantic-language/`

What The Runtime Actually Does

At runtime, MotionMixApp's `EchelonBridge` owns:

• a Rust brain handle;
• a Rust audio handle;
• a `SANService`;
• a `ClaimBridgeService`;
• cached pose, mocopi, pocket IMU, and watch features.

Its `step(dt:)` loop drains sensor frames, steps the Rust Echelon brain, bridges
brain state to audio, reads latent/lexicon/UI outputs, runs SAN, and then calls
ClaimBridge on Rust dynamics.

That means the current deployed spine is:

sensors
  -> EchelonBridge
  -> Rust EchelonCore
  -> latent / lexicon / UI state
  -> SANService
  -> audio parameters / trajectory logging
  -> ClaimBridgeService
  -> N'Ko-style motion claim inscription

The docs should not collapse this into a single "LIM-RPS" model. `LIM-RPS`
exists in older/research code, but the current source names the production
runtime around `EchelonCore`, `LatentUpdater`, `SANPipeline`, `DiffusionService`,
and `ClaimBridge`.

What Is Generated

Music:

- Rust Echelon audio and MotionMix audio services respond to latent/lexicon/SAN
output.
- SAN exists as a Rust pipeline loaded through `SANService`.
- Current docs must not claim a specific training set or validation loss unless
the training report is present.

Visuals:

- Unity/LUME consumes body/skeleton/motion feeds.
- Mac4 currently receives Sony mocopi and feeds Unity locally.
- MotionMix BodyTruth is the intended shared body-state authority, but it has
had stability issues and should be described as an integration layer, not as a
universally stable source until verified.

DJ control:

- K11 owns Rekordbox safety.
- AirDeck/gesture detection should send intent through K11's bridge, not direct
Unity-to-Rekordbox commands.
- Camera-only hand raise can work without mocopi; mocopi is an accuracy source,
not a hard dependency for every gesture.

N'Ko / inscription:

• ClaimBridge is implemented.
• It maps Echelon dynamics to controlled motion claim signals.
• It does not mean every gesture has full natural N'Ko phrasing yet.

Diffusion Status

The old wording was too strong.

MotionMixApp contains `DiffusionService.swift`, and the implemented CoreML path is:

128D-ish dynamics assembly
  -> temporary 104D truncation for ConditioningEncoder
  -> ConditioningEncoder.mlmodelc -> 768D embedding
  -> optional TTTAdaptationLayer
  -> FlowGenerator1Step.mlmodelc
  -> token logits / TokenGrid

This is best described as a conditioned one-step flow/token generator path with
fallbacks. It is not proven by the Swift code to be a full multi-step DDPM/DDIM
diffusion sampler. Comp-Core contains research diffusion modules, but those are
not the same thing as the currently deployed MotionMixApp generation path.

What This Is Not

- It is not just motion graphics. Body state can drive audio, visuals, camera,
and inscription.
- It is not a fully trained end-to-end body-to-music foundation model.
- It is not a single canonical `LIM-RPS` architecture.
- It is not a verified full 128D CoreML conditioning path yet.
- It is not a claim that SAN V5/V6 has a specific dataset size or validation loss
unless that artifact is tied to the current local files.

Direction

The near-term direction is:

1. Make the source-of-truth state explicit.
2. Keep body capture and training capture structured.
3. Stabilize MotionMix BodyTruth as the shared bus.
4. Keep K11 as the DJ command safety gate.
5. Use mocopi, cameras, phone/watch sensors, and pose viewers as complementary
evidence sources.
6. Promote only verified training and model claims into the public docs.