🕺 Computational Choreography Integration Plan
``` ┌────────────────────────────────────────────────────────────────────┐ │ THE BUFF BARISTA LOOP │ ├────────────────────────────────────────────────────────────────────┤ │ │ │ ┌─────────┐ ┌──────────────┐ ┌─────────────┐ │ │ │ MUSIC │ ───> │ CC-MOTIONGEN │ ───> │ CHOREOGRAPHY │ │ │ │ (Input) │ │ (Diffusion) │ │ (Generated) │ │ │ └─────────┘ └──────────────┘ └──────┬──────┘ │ │ │ │ │ ▼ │ │ ┌──────────────┐ │ │ │ PRACTICE │ │ │ │ (iPhone App) │ │ │ └──────┬───────┘ │ │ │ │ │ ┌───────────────────────────────────────
Full Public Reader
🕺 Computational Choreography Integration Plan
Vision: Auto-generate Zumba choreography using AI + body motion → real-time music generation
---
🎯 The Dream
┌────────────────────────────────────────────────────────────────────┐
│ THE BUFF BARISTA LOOP │
├────────────────────────────────────────────────────────────────────┤
│ │
│ ┌─────────┐ ┌──────────────┐ ┌─────────────┐ │
│ │ MUSIC │ ───> │ CC-MOTIONGEN │ ───> │ CHOREOGRAPHY │ │
│ │ (Input) │ │ (Diffusion) │ │ (Generated) │ │
│ └─────────┘ └──────────────┘ └──────┬──────┘ │
│ │ │
│ ▼ │
│ ┌──────────────┐ │
│ │ PRACTICE │ │
│ │ (iPhone App) │ │
│ └──────┬───────┘ │
│ │ │
│ ┌─────────────────────────────────────────┘ │
│ │ │
│ ▼ │
│ ┌──────────┐ ┌──────────────┐ ┌───────────────┐ │
│ │ MOTION │ ──> │ ECHELON │ ──> │ LIVE MUSIC │ │
│ │ (iPhone) │ │ ADAPTER │ │ (Generated) │ │
│ └──────────┘ └──────────────┘ └───────────────┘ │
│ │ │
│ └─── Dancing with weights (4-6 lbs) ───────────┘ │
│ │
└────────────────────────────────────────────────────────────────────┘---
📦 Comp-Core Assets
### Motion Layer (Rust)
| Component | Purpose | Status |
|-----------|---------|--------|
| `cc-types` | Core motion types (MotionWindow, SkeletonFrame) | ✅ FROZEN |
| `cc-anticipation` | Anticipation kernel for choreo | ✅ Complete |
| `cc-collection` | Sensor fusion via EKF | ✅ Complete |
| `cc-gesture` | Gesture recognition | 🔄 Partial |
| `cc-window-aligner` | Motion alignment (5-stage) | ✅ 99.5
### ML Layer (Python)
| Component | Purpose | Status |
|-----------|---------|--------|
| `cc-ml/cc_motiongen` | Diffusion-based motion generation | ✅ Substantial |
| `diffusion/` | UNet1D (116M params) | ✅ Trained |
| `motionphrase/` | Semantic phrase library | 🔄 Building |
| `pattern_coder/` | Motion pattern encoding | ✅ Complete |
### Audio Layer
| Component | Purpose | Status |
|-----------|---------|--------|
| `echelon_adapter.py` | Motion → Music control mapping | ✅ Complete |
| `choreo_server.py` | Real-time choreo via ZMQ | ✅ Complete |
| `LIM-RPS` | 3-timescale latent state solver | ✅ Complete |
| Strudel IR | Music generation DSL | ✅ Integrated |
---
🔄 Integration Pipeline
Phase 1: Choreography Generation (Music → Motion)
INPUT: Zumba song (MP3/WAV)
↓
STEP 1: Audio feature extraction (librosa)
- MFCCs, spectral features, beat tracking
↓
STEP 2: CC-MotionGen diffusion (20 DDIM steps)
- 25-dimensional motion output at 30fps
- Position, velocity, orientation, phase, style
↓
STEP 3: Post-processing
- Temporal smoothing
- Style transfer (Zumba vs Salsa vs Breakdance)
↓
OUTPUT: Choreography sequence (JSON/Lottie)Phase 2: Practice Mode (Motion → Feedback)
INPUT: Your motion via iPhone (CMMotionManager)
↓
STEP 1: cc-collection (sensor fusion)
- Accelerometer + Gyroscope fusion via EKF
↓
STEP 2: cc-window-aligner (5-stage pipeline)
- Time normalize → Resample → Coordinate unify
↓
STEP 3: Compare to generated choreography
- DTW alignment
- Accuracy scoring
↓
OUTPUT: Real-time feedback (haptic + visual)Phase 3: Live Performance (Motion → Music)
INPUT: Your dancing (with 4-6 lb weights)
↓
STEP 1: cc-anticipation
- Extract: commitment, uncertainty, transition_pressure, novelty
↓
STEP 2: LIM-RPS v0 solver
- 3-timescale latent state (fast/medium/slow)
↓
STEP 3: EchelonAdapter
- Map state to music controls:
• tempo_nudge (±5%)
• swing_amount (0-1)
• intensity (0-1)
• drop_trigger (bool)
↓
STEP 4: Strudel IR → Audio
↓
OUTPUT: Real-time generated music---
🎵 Zumba Music Catalog Integration
### Supported Genres (for generation)
- Reggaeton (BPM: 90-110)
- Salsa (BPM: 160-220)
- Cumbia (BPM: 80-100)
- Merengue (BPM: 120-160)
- Bachata (BPM: 120-140)
- Afrobeats (BPM: 100-120)
### Training Data Needed
- [ ] 50+ labeled Zumba routines (video + audio)
- [ ] Motion capture of signature moves
- [ ] Style embeddings per genre
---
💪 Weighted Movement Adaptation
### The Buff Barista Signature
- Light weights: 4-6 lbs
- Arm emphasis movements
- Controlled, powerful gestures
Weight-Aware Motion Model
# Conceptual: weight_factor affects motion dynamics
motion = cc_motiongen.generate(
audio=song,
style="zumba",
weight_factor=0.3, # 0=no weights, 1=heavy emphasis
emphasis=["arms", "shoulders", "core"]
)---
🛠️ Implementation Roadmap
### Sprint 1: Foundation (Week 1-2)
- [ ] Set up CC-MotionGen inference pipeline
- [ ] Test with sample Zumba tracks
- [ ] Validate motion output format
- [ ] Create visualization tool (Lottie or Three.js)
### Sprint 2: Practice App (Week 3-4)
- [ ] iPhone motion capture integration
- [ ] Real-time comparison to generated choreo
- [ ] Accuracy scoring system
- [ ] Basic haptic feedback
### Sprint 3: Live Music (Week 5-6)
- [ ] EchelonAdapter tuning for Zumba
- [ ] Weight-aware motion → music mapping
- [ ] Test with Strudel patterns
- [ ] Latency optimization (<50ms)
### Sprint 4: Polish (Week 7-8)
- [ ] Style transfer between genres
- [ ] Choreography library (save/load routines)
- [ ] Share routines via QR code
- [ ] Record performance videos with overlay
---
🎯 Success Metrics
| Metric | Target | How to Measure |
|---|---|---|
| Motion generation latency | <200ms | Time from audio chunk to motion output |
| Music generation latency | <50ms | Time from motion to audio output |
| Choreography accuracy | >80 | |
| Genre recognition | >90 | |
| Weight adaptation | Visible difference | A/B comparison with/without weight mode |
---
📱 App Concept: "Buff Flow"
### Screens
1. Library — Your Zumba songs + generated choreos
2. Generate — Drop a song, get choreography
3. Practice — Follow along, get scored
4. Perform — Motion → music (live mode)
5. Profile — Progress, favorite routines
### Tech Stack
- SwiftUI (iOS app)
- Core Motion (motion capture)
- CC-Core via Swift bindings or WebSocket
- On-device ML (Core ML export of motiongen)
---
🔗 Related Files
| File | Location |
|---|---|
| CC-MotionGen | `Desktop/Comp-Core/core/ml/cc-ml/cc_motiongen/` |
| Echelon Adapter | `Desktop/Comp-Core/core/runtime/cc-core/cc_core/realtime/echelon_adapter.py` |
| Choreo Server | `Desktop/Comp-Core/core/runtime/cc-core/cc_core/realtime/choreo_server.py` |
| Motion Layer | `Desktop/Comp-Core/core/motion/` |
| Buf Barista Profile | `Desktop/spine/Buf Barista/PROFILE.md` |
---
🚀 Quick Start
# Generate choreography from a song
python -m cc_motiongen.inference.generate \
--audio [home-path] \
--style zumba \
--output choreo.json
# Start real-time choreo server
python -m cc_core.realtime.choreo_server \
--bind tcp://[ip]:5555
# Run motion → music loop
python -m cc_core.realtime.echelon_adapter \
--input-socket tcp://[ip]:5555 \
--output-osc [ip]:7000---
Created: January 31, 2026
For: Xanadu Launch Preparation
Promotion Decision
Attach run IDs, datasets, metrics, and reproduction commands.
Source Anchor
spine/Buf Barista/COMPUTATIONAL-CHOREOGRAPHY.md
Detected Structure
Method · Evaluation · Code Anchors · Architecture