Motion Control Studio — Architecture Aligned to Computational Choreography & Echelon

Motion Control Studio — Architecture Aligned to Computational Choreography & Echelon

## What This Is (and Isn’t)
- Instrument-grade control surface that routes embodied input (phone/watch/IMU) into a real-time action layer with <10 ms reflex latency.
- Computational choreography-native: built around `cc-core` (LIM‑RPS/DELL) equilibria, not hardcoded gesture thresholds.
- Plugin‑first & offline‑first: everything runs locally; cloud sync is optional and never required for performance.
- Echelon optional (future): today it can run standalone with the action adapters; when Echelon lands, we swap the execution sink without redesigning the control plane.

## The Missing Layer We Add Now
Introduce an explicit Embodied Latent & Choreography Engine between fusion and gestures. This is cc-core (LIM‑RPS/DELL) producing a shared latent “moment” that all consumers read, so we do not rebuild Echelon at lower power or bypass computational choreography.

Stack at a Glance (mapped to this repo)

Tauri shell (Rust) + Next.js UI (apps/desktop)           <-- cc-studio front-end host
        │
        ├─ UI (React/TS, Tailwind v4): calibration, mappings, live viz
        ├─ IPC: Tauri commands + events
        ▼
Rust control backend (apps/desktop/src-tauri)            <-- binds to cc-core + cc-echelon
        ├─ Sensor bridge: phone/watch/IMU over WS/BT/serial
        ├─ Fusion + DELL runtime: cc-core exported to ONNX/StableHLO or Rust port
        ├─ Gesture/intent: template matching + classifier + action mapper
        ├─ Control bus: publishes FastState/SlowState/telemetry
        └─ Action adapters: keyboard (enigo), MIDI/OSC (midir/rosc), HTTP
               │
               ▼
Echelon engine (cc-echelon/*)                             <-- quantized execution
        ├─ Scheduler/Link clock, safety policies, action queue
        ├─ Audio graph, FX, phrase intelligence
        └─ UI shell (egui) for deck lanes/automation if running headful

Supporting packages:
- `cc-core/` — JAX/Flax equilibria (LIM‑RPS + DELL, learned geometry, implicit diff). Export models to ONNX/StableHLO for Rust inference.
- `cc-studio/` — existing Python runtime utilities (logging, DJ agent glue); keep as sidecars until everything is ported.
- `configs/` — gesture/mapping/profile TOML/JSON; keep user profiles here so Studio + Echelon share the same contract.

## Computational Choreography Contract (what the runtime must honor)
- Two time scales:
- Fast loop (frame rate 100–200 Hz): LIM‑RPS/DELL fast `x*` for reflex control; target <4 iterations, <5 ms end-to-end including IPC.
- Slow loop (beat rate 2–4 Hz): DELL slow `y*` for phrase/scene control; scheduled on beat using Link clock; keep coupling energy φ stable.
- Phase-aware control: expose ψ from the phase head; UI shows ψ vs θ (beat phase) and re-lock time after dropouts.
- Contractive operators: retain cc-core spectral-norm constraints and step fields; log contraction ratio/headroom in telemetry.
- Fallbacks: if DELL stalls, hold last `x`/`y`, keep template matcher alive, and keep scheduler safe (never block audio).

## Runtime Dataflow
1) Sensors → Fusion: phone/watch/IMU arrive via WS/BT/serial → packed into fixed-shape buffers → fusion (Kalman/filters).
2) Fusion → Choreography (cc-core): cc-core equilibria (fast/slow) produce the shared latent moment `ChoreoMoment` (`x*`, `ψ`, `φ`, limb energies, confidence).
3) Choreography → Gesture/Pattern (operating on latent, not raw IMU):
- ML classifier (ONNX) on sequences of `x*`,
- DTW/template matcher on latent trajectories,
- Motif/phrase detectors on equilibrium metrics.
4) Intent → Actions/Hosts:
- Map to `ActionDefinition` (keyboard/MIDI/OSC/HTTP/custom plugin).
- Quantize via local scheduler (beat-safe) or, when available, Echelon scheduler.
5) Execution & Feedback: adapters or Echelon execute; telemetry (latency, residuals, φ, ψ, headroom) published to UI.

## Operating Without Echelon (today)
- Execution sink: use built-in adapters (keyboard/enigo, MIDI/OSC, HTTP) for actions; scheduler is a local quantizer (beat-aware, deck-safe when applicable).
- Fusion/gesture: cc-core ONNX/StableHLO for fast/slow equilibria; classifier + template matcher for redundancy.
- Persistence: write profiles/mappings to `configs/`; log telemetry locally for rehearsal/replay.
- Upgrade path: isolate the “sink” so replacing the executor with Echelon later is a wiring change, not a redesign.

## Echelon Integration Points (future-proof, but not required now)
- Control bus types: if/when Echelon is present, reuse `cc-echelon/crates/control-bus` structs for `FastState`, `SlowState`, `Telemetry`, `Intent`, and `Action`.
- Scheduler API: swap the local quantizer with `cc-echelon/crates/scheduler` for beat-quantized dispatch and safety policies.
- Phrase intelligence: optional—route motion context into `crates/media` phrase recommender; UI shows suggested clips/mappings.
- Sidecars: ASR (whisper-rs) and ANN/diffusion stay as side processes; communicate over shmem/IPC, never in the audio thread.

## Shared Types (align Rust/TS/Python)
- `ChoreoFrame`: fused sensor snapshot (timestamp, per-device IMU/pose features, optional audio features).
- `ChoreoMoment`: `x*` (latent), `ψ` (phase), `φ` (coupling energy), per-limb energies, beat/tempo, confidence.
- Events: `sensor-frame` in → `choreo-moment` out; gestures/plugins subscribe to `choreo-moment`.
- Plugin hook: `onChoreoMoment(moment: ChoreoMoment)` so hosts (including Echelon) consume the same latent without duplicate pipelines.

## UI/UX Requirements (beyond generic dashboards)
- Calibration flows for each device class (phone, watch, VR, IMU array) with saved offsets per profile.
- Live “agreement” view: ψ vs θ wheel, φ meter, residual timeline, device health.
- Mapping editor that writes to `configs/mappings/*.toml` (action + plugin registry).
- Plugin surfaces: settings/visualizer hooks per MotionPlugin, plus OS-level permissions helper (macOS accessibility for keyboard control).
- Performance HUD: end-to-end latency (sensor→action), frame drop counters, headroom (1 − σ_max).

## Performance Guardrails
- Reflex path budget: ≤10 ms sensor→action; target 4–6 ms.
- Audio safety: no allocations/locks in `cc-echelon` callback; control bus is SPSC ringbuffer; drop-on-full instead of blocking.
- IPC: prefer shared memory for frame streams; WS only for UI; avoid JSON in hot path (use bincode/flatbuffers).
- Profiling: log latency percentiles, contraction ratios, mask violations; surface overruns in UI.

## Redundancy & Safety (avoid single points of failure)
- Dual gesture path: ML classifier and DTW/template matcher; if ML stalls, template continues.
- Dual equilibria: if slow loop misses a beat, hold last `y`; if fast loop skips a solve, hold last `x`.
- Action gating: all outbound actions pass through a safety/mask layer; panic kill-switch to drop all actions.
- Offline-first: no cloud dependency for runtime; model/asset preload at startup; sensors auto-reconnect without blocking the loop.

## Implementation Path (bridging today’s repos)
1) Model export: export cc-core DELL fast/slow + classifiers to ONNX/StableHLO; add a small loader in `apps/desktop/src-tauri`.
2) Choreography service: wrap cc-core (Python IPC first) to emit `ChoreoMoment`; bake type alignment (Rust/TS/Python).
3) Tauri backend: sensor bridges → choreography service → gesture/pattern layer → local scheduler → action adapters; expose Tauri commands/events for UI.
4) Echelon hook-up (later): swap the sink to Echelon via control-bus + scheduler when ready; keep the choreography layer intact.
5) UI slices: build calibration, mapping, live viz, and plugin panes in Next.js; wire to Tauri IPC and `choreo-moment` events.
6) Telemetry: publish ψ, φ, residuals, headroom, latency to UI + logs; align field names with `cc-core/metrics`.
7) Hardening: offline mode, profile management, auto-reconnect for sensors, and panic/kill switches for actions.

## cc-core Runtime Modes (make this explicit)
- Mode 1 — Research (today): Rust backend ↔ Python cc-core over IPC (ZMQ/gRPC/custom). Use for rapid LIM‑RPS/DELL changes and tuning.
- Mode 2 — Production Alpha (soon): Export equilibria to ONNX/StableHLO → run via ORT in Rust; no Python dependency for Tauri packaging.
- Mode 3 — Echelon Unified (later): Embed cc-core inference alongside Echelon’s audio/scheduler for sample-accurate, beat-locked control.

## Control Bus & Clock Authority
- Types (define now): `FastState`, `SlowState`, `Telemetry`, `Intent`, `Action`, `ChoreoMoment`. Use identical field names across Rust/TS/Python.
- Clock ownership: If Echelon is present, follow its Link session; otherwise Motion Control Studio owns the Link clock/beat phase. Slow-loop scheduling uses that clock in both cases.

That’s the architecture we should build: a Tauri/Next control plane that is choreography-native (cc-core), execution-safe (Echelon), and instrument-grade (plugin and profile aware), not a generic gesture app.