Computational Choreography

Computational Choreography

The Discipline

Version: 1.1.0
Last Updated: 2025-01-01
Status: Foundational
Parent: [00-OVERVIEW.md](00-OVERVIEW.md)

---

Definition

Computational Choreography is the practice of treating embodied human motion as a semantic object capable of generating, modifying, and controlling digital experiences. It is not motion capture, not gesture recognition, and not animation—though it can incorporate all three.

At its core, Computational Choreography asks:

> What does movement mean, and how can we compute that meaning in real time?

This is the governing philosophy for the entire Comp-Core system. Every architectural decision, every algorithm choice, and every data structure is evaluated against this question.

---

Core Principles

1. Motion is Semantic

Movement carries information beyond its kinematic description. A raised arm is not just a position vector; it is an intention, a signal, a communication.

Implications:
- We extract meaning, not just position
- We model intention, not just trajectory
- We preserve expressivity, not just accuracy

Implementation: See [06-ANTICIPATION.md](06-ANTICIPATION.md) for how we extract semantic signals.

2. Anticipation Over Reaction

Traditional systems react to motion events after they occur. Computational Choreography anticipates motion before it completes. By analyzing commitment, uncertainty, and transition pressure, the system can act on intention rather than completion.

Implications:
- Actions fire when motion becomes irreversible, not when it ends
- The system prepares responses while motion is still in progress
- Latency is reduced because we don't wait for completion

Implementation: See [06-ANTICIPATION.md](06-ANTICIPATION.md) for the commitment/uncertainty model.

3. Embodied, Not Abstract

The system does not process motion as abstract data. It maintains awareness of:

Implementation: See [05-SENSOR_FUSION.md](05-SENSOR_FUSION.md) for physical modeling.

4. Continuous, Not Discrete

Computational Choreography rejects the discrete gesture paradigm. Instead of recognizing isolated gestures, it processes continuous motion fields. Gestures emerge from the field; they are not imposed upon it.

Implications:
- No fixed gesture vocabulary required
- Motion is always being processed, not just during "gestures"
- Classification is continuous, with confidence varying smoothly

Implementation: See [07-GESTURE_RECOGNITION.md](07-GESTURE_RECOGNITION.md) for continuous classification.

---

The Discipline Stack

┌─────────────────────────────────────────────────────────────┐
│                                                             │
│              COMPUTATIONAL CHOREOGRAPHY                     │
│                    The Discipline                           │
│                                                             │
│   "What does movement mean, and how can we compute it?"     │
│                                                             │
├─────────────────────────────────────────────────────────────┤
│                                                             │
│                    TrajectoryOS                             │
│               The Operating System                          │
│                                                             │
│  Time Scale: Hours → Years                                  │
│  Question: "What does this moment mean in context?"         │
│  Output: Memory, context, style signature                   │
│                                                             │
│  Components:                                                │
│    • RAG++ (5D trajectory memory)                          │
│    • Orbit (session orchestration)                          │
│    • CognitiveTwin (style learning)                        │
│                                                             │
│  See: 02-TRAJECTORY_OS.md                                  │
│                                                             │
├─────────────────────────────────────────────────────────────┤
│                                                             │
│                      Echelon                                │
│                The Real-Time Engine                         │
│                                                             │
│  Time Scale: Milliseconds → Seconds                         │
│  Question: "What is happening now, and how do we respond?" │
│  Output: Control signals, gestures, audio                   │
│                                                             │
│  Components:                                                │
│    • cc-collection (sensor fusion)                         │
│    • cc-anticipation (commitment/uncertainty)               │
│    • cc-gesture (classification)                           │
│    • DELL (dual-equilibrium dynamics)                      │
│    • cc-conductor (beat scheduling)                        │
│                                                             │
│  See: 03-ECHELON.md                                        │
│                                                             │
└─────────────────────────────────────────────────────────────┘

---

Conceptual Vocabulary

These terms are precisely defined and used consistently throughout the codebase.

Commitment

The degree to which a motion has become irreversible. High commitment means the body has committed to a trajectory; the motion will complete.

Range: [0, 1]
Rust Type: `Commitment(f32)` (newtype wrapper)
Computed By: [cc-anticipation](06-ANTICIPATION.md)

Uncertainty

The number of plausible futures that remain. High uncertainty means many possible continuations; low uncertainty means the trajectory is constrained.

Range: [0, 1]
Rust Type: `Uncertainty(f32)`
Computed By: [cc-anticipation](06-ANTICIPATION.md)

Transition Pressure

The rate at which futures are collapsing. Positive pressure means options are narrowing (motion converging); negative pressure means options are opening (motion diverging).

Range: [-∞, +∞]
Rust Type: `TransitionPressure(f32)`
Computed By: `dC/dt + α * (-dU/dt)`

Recovery Margin

The distance to balance or attractor loss. High recovery margin means the body is stable; low recovery margin means instability is imminent.

Range: [0, 1]
Computation: `1.0 - max(balance_violation, joint_limit_violation, speed_saturation)`

Phase Stiffness

How locked the motion is to an internal metronome. High phase stiffness means rhythmic, periodic motion; low phase stiffness means arrhythmic motion.

Range: [0, 1]
Computation: `periodicity * (1.0 - angular_jerk)`

Novelty

Distance from recent motion regimes. High novelty means exploring new territory; low novelty means repeating familiar patterns.

Range: [0, 1]
Computation: `distance(current_embedding, centroid(history)) / 2.0`

Stability

Local stationarity of dynamics. High stability means motion is predictable; low stability means motion is chaotic.

Range: [0, 1]
Computation: `1.0 - latent_velocity_norm / max_expected_velocity`

---

The Anticipation Model

Unlike reactive systems that trigger on gesture completion, Computational Choreography uses an anticipation model:

flowchart LR
    subgraph Motion [Motion Field]
        Sensor[Sensor Data<br/>60 Hz]
        Window[Motion Window<br/>1s @ 50 Hz]
    end

    subgraph Features [Feature Extraction]
        Kin[Kinematics<br/>Position, Velocity, Acceleration]
        Lat[Latent Space<br/>Regime Embedding]
    end

    subgraph Scalars [Anticipation Scalars]
        C[Commitment<br/>∈ [0,1]]
        U[Uncertainty<br/>∈ [0,1]]
        T[Transition Pressure<br/>∈ [-∞,+∞]]
    end

    subgraph Action [Action Generation]
        Conductor[cc-conductor<br/>Beat Quantization]
        Audio[Audio Output<br/>48kHz]
    end

    Sensor --> Window --> Kin --> C
    Window --> Lat --> U
    C --> Conductor
    U --> Conductor
    T --> Conductor
    Conductor --> Audio

Decision Space

The key insight is that commitment and uncertainty provide a two-dimensional space for decision-making:

         ↑ Uncertainty
         │
    1.0  │  SURPRISING     │     WAITING
         │  (adapt fast)   │   (observe)
         │                 │
    0.5  ├─────────────────┼─────────────────
         │                 │
         │  COMMITTED      │   PREPARING
         │  (execute)      │ (ready response)
         │                 │
    0.0  └─────────────────┴─────────────────→
                          0.5              1.0  Commitment

---

Relation to Other Paradigms

Motion Capture

Gesture Recognition

Animation

Musical Performance

---

Design Philosophy

1. No Magic Numbers

All thresholds and parameters must be:
- Documented with their purpose
- Tunable at runtime
- Validated against physical constraints

// Bad: unexplained constant
if commitment > 0.7 { fire_gesture() }

// Good: documented, configurable
if commitment.is_high(config.commitment_threshold) {
    // Threshold validated in config: must be > 0.5 for stability
    fire_gesture()
}

2. Deterministic Replay (INV-001)

Given the same sensor input, the system must produce identical output. This enables:
- Debugging (replay problematic sessions)
- Testing (regression tests)
- Training data generation (augment dataset)

Implementation: No random seeds, no wall-clock time in processing, seeded RNG only for non-deterministic features.

3. Graceful Degradation

When sensors fail or coverage drops, the system must:
1. Detect the degradation
2. Fall back to available data
3. Communicate the degradation state
4. Recover when sensors return

Implementation: See [05-SENSOR_FUSION.md](05-SENSOR_FUSION.md) for occlusion handling.

4. Type Safety

Domain-specific newtypes for all semantic values:

// Bad: raw floats (no semantic meaning, no validation)
fn process(commitment: f32, uncertainty: f32) { ... }

// Good: newtypes with validation (meaning and bounds enforced)
fn process(commitment: Commitment, uncertainty: Uncertainty) { ... }

All newtypes validate bounds on construction and provide semantic methods.

---

Implementation in Comp-Core

Computational Choreography is implemented across multiple crates:

Each crate implements a layer of the discipline, from raw sensor data to musical expression.

---

Philosophical Foundations

Computational Choreography draws from:

1. Embodied Cognition: The body is not separate from cognition; movement is thought
2. Ecological Psychology: Perception is for action; affordances shape behavior
3. Dynamical Systems: Motion is a trajectory through state space, not a sequence of poses
4. Phenomenology: First-person experience of movement matters

These inform the design:
- We track dynamics, not just position
- We model intention, not just trajectory
- We preserve expressivity, not just accuracy
- We enable flow, not just control

---

Further Reading

• [02-TRAJECTORY_OS.md](02-TRAJECTORY_OS.md) — The long-horizon operating system
• [03-ECHELON.md](03-ECHELON.md) — The real-time engine
• [06-ANTICIPATION.md](06-ANTICIPATION.md) — Technical details of anticipation computation
• [07-GESTURE_RECOGNITION.md](07-GESTURE_RECOGNITION.md) — How gestures emerge from anticipation

---

Revision History