Grand Diomande Research · Full HTML Reader

Audio Engine

**Version**: 1.1.0 **Last Updated**: 2025-01-01 **Status**: Production **Parent**: [03-ECHELON.md](03-ECHELON.md) **Previous**: [07-GESTURE_RECOGNITION.md](07-GESTURE_RECOGNITION.md) **Crate**: `core/cc-echelon/crates/cc-conductor/` **Tests**: 31 passing

Embodied Trajectory Systems architecture technical paper candidate score 40 .md

Full Public Reader

Audio Engine

Layer 5-7: Beat Scheduling and DSP

Version: 1.1.0
Last Updated: 2025-01-01
Status: Production
Parent: [03-ECHELON.md](03-ECHELON.md)
Previous: [07-GESTURE_RECOGNITION.md](07-GESTURE_RECOGNITION.md)
Crate: `core/cc-echelon/crates/cc-conductor/`
Tests: 31 passing

---

Overview

The audio engine provides beat-quantized scheduling, real-time DSP, and DJ software integration. It operates under strict latency constraints where every millisecond matters.

Latency Budget

StageBudgetNotes
Gesture detection500μsFrom cc-gesture
Action scheduling100μsQueue insertion
Beat quantization50μsNext beat calculation
MIDI/OSC send200μsNetwork I/O
Audio callback5.3ms256 samples @ 48kHz
Total< 6msEnd-to-end

---

Architecture

mermaid
flowchart TB
    subgraph Input [Input]
        Lexicon[Lexicon Signals]
        Gesture[Gesture Events]
        Link[Ableton Link]
    end

    subgraph Conductor [cc-conductor]
        Tempo[Tempo Tracker]
        Quantizer[Beat Quantizer]
        Actions[Action Queue]
    end

    subgraph Scheduler [scheduler]
        Timeline[Beat Timeline]
        Pending[Pending Actions]
        Fire[Fire Actions]
    end

    subgraph Engine [audio-engine]
        DSP[DSP Chain]
        Mixer[Mixer]
        Output[Audio Output]
    end

    subgraph DJ [DJ Integration]
        RB[Rekordbox]
        Serato[Serato]
        MIDI[MIDI/OSC]
    end

    Lexicon --> Tempo
    Gesture --> Actions
    Link --> Tempo
    Tempo --> Quantizer --> Actions --> Pending
    Pending --> Fire --> DSP --> Mixer --> Output
    Fire --> MIDI --> RB
    Fire --> MIDI --> Serato

---

cc-conductor

Orchestrates motion-to-music translation:

rust
pub struct Conductor {
    config: ConductorConfig,
    tempo: TempoTracker,
    section: SectionState,
    pattern_engine: PatternEngine,
    action_queue: VecDeque<ScheduledAction>,
}

impl Conductor {
    pub fn process(
        &mut self,
        lexicon: &Lexicon,
        gesture: Option<&GestureEvent>,
        dt: f32,
    ) -> Vec<Action> {
        // Update tempo from lexicon
        self.tempo.update(lexicon.energy, dt);

        // Update section state
        self.section = self.compute_section(lexicon);

        // Generate pattern modifications
        let pattern_changes = self.pattern_engine.process(lexicon, self.section);

        // Process gestures into actions
        let gesture_actions = gesture.map(|g| self.gesture_to_action(g));

        // Collect and quantize actions
        self.collect_actions(pattern_changes, gesture_actions)
    }
}

TempoTracker

Smoothly adapts tempo to motion energy:

rust
pub struct TempoTracker {
    base_bpm: f32,          // 120.0
    current_bpm: f32,
    target_bpm: f32,
    smoothing: f32,         // 0.95 (high smoothing)
    min_bpm: f32,           // 80.0
    max_bpm: f32,           // 160.0
}

impl TempoTracker {
    pub fn update(&mut self, energy: f32, dt: f32) {
        // Map energy [0,1] to tempo range
        self.target_bpm = self.min_bpm + energy * (self.max_bpm - self.min_bpm);

        // Smooth transition
        self.current_bpm = self.smoothing * self.current_bpm
                         + (1.0 - self.smoothing) * self.target_bpm;
    }

    pub fn beat_duration(&self) -> f32 {
        60.0 / self.current_bpm
    }
}

---

Beat Scheduler

Quantizes actions to the beat grid:

rust
pub struct BeatScheduler {
    timeline: BeatTimeline,
    pending: Vec<ScheduledAction>,
    quantize_mode: QuantizeMode,
}

pub enum QuantizeMode {
    Off,           // Fire immediately
    Beat,          // Quantize to beat
    Bar,           // Quantize to bar
    Phrase,        // Quantize to 4/8 bar phrase
}

pub struct ScheduledAction {
    pub action: Action,
    pub fire_at: BeatPosition,
    pub priority: Priority,
}

impl BeatScheduler {
    pub fn schedule(&mut self, action: Action, quantize: QuantizeMode) {
        let fire_at = match quantize {
            QuantizeMode::Off => self.timeline.now(),
            QuantizeMode::Beat => self.timeline.next_beat(),
            QuantizeMode::Bar => self.timeline.next_bar(),
            QuantizeMode::Phrase => self.timeline.next_phrase(),
        };

        self.pending.push(ScheduledAction {
            action,
            fire_at,
            priority: Priority::Normal,
        });
    }

    pub fn tick(&mut self, now: BeatPosition) -> Vec<Action> {
        let (ready, pending): (Vec<_>, Vec<_>) = self.pending
            .drain(..)
            .partition(|a| a.fire_at <= now);

        self.pending = pending;
        ready.into_iter().map(|a| a.action).collect()
    }
}

BeatTimeline

Integrates with Ableton Link for network sync:

rust
pub struct BeatTimeline {
    link: Option<AbletonLink>,
    local_tempo: f32,
    beat_phase: f32,
    bar_phase: f32,
}

impl BeatTimeline {
    pub fn update(&mut self, dt: f32) {
        if let Some(ref link) = self.link {
            // Sync from Link
            let state = link.capture_audio_session_state();
            self.beat_phase = state.beat_at_time(link.clock_micros()) % 1.0;
            self.bar_phase = state.beat_at_time(link.clock_micros()) % 4.0 / 4.0;
        } else {
            // Local timing
            self.beat_phase = (self.beat_phase + dt / self.beat_duration()) % 1.0;
            self.bar_phase = (self.bar_phase + dt / (self.beat_duration() * 4.0)) % 1.0;
        }
    }

    pub fn next_beat(&self) -> BeatPosition {
        BeatPosition::from_phase(1.0 - self.beat_phase)
    }
}

---

Real-Time DSP

Hard real-time audio processing with zero-allocation callbacks:

rust
pub struct AudioEngine {
    sample_rate: u32,       // 48000
    buffer_size: usize,     // 256
    channels: Vec<Channel>,
    master: MasterChannel,
}

impl AudioEngine {
    /// Audio callback - MUST be lock-free, allocation-free
    pub fn process(&mut self, output: &mut [f32]) {
        // Clear output
        for sample in output.iter_mut() {
            *sample = 0.0;
        }

        // Mix channels
        for channel in &mut self.channels {
            if channel.is_active() {
                channel.render_into(output);
            }
        }

        // Master processing
        self.master.process(output);
    }
}

Safety Rules

RuleEnforcement
No allocationPre-allocated buffers
No blockingLock-free queues
No syscallsAvoid I/O in callback
Bounded timeFixed buffer size

---

DJ Integration

Action Types

rust
pub enum Action {
    // Transport
    Play,
    Pause,
    CueSet,
    CueGoto,

    // Mixing
    CrossfaderLeft(f32),
    CrossfaderRight(f32),
    EqLow(u8, f32),     // (deck, value)
    EqMid(u8, f32),
    EqHigh(u8, f32),

    // Effects
    EffectOn(u8),       // effect slot
    EffectOff(u8),
    EffectDry(u8, f32),

    // Loops
    LoopIn,
    LoopOut,
    LoopDouble,
    LoopHalf,

    // Navigation
    JumpForward(f32),   // beats
    JumpBack(f32),
}

Rekordbox Bridge

rust
pub struct RekordboxBridge {
    midi_out: MidiOutput,
    key_mappings: HashMap<Action, MidiMessage>,
}

impl RekordboxBridge {
    pub fn send(&mut self, action: Action) -> Result<()> {
        if let Some(msg) = self.key_mappings.get(&action) {
            self.midi_out.send(msg)?;
        }
        Ok(())
    }
}

Serato Bridge

rust
pub struct SeratoBridge {
    osc_client: OscClient,
    address_mappings: HashMap<Action, String>,
}

impl SeratoBridge {
    pub fn send(&mut self, action: Action) -> Result<()> {
        if let Some(addr) = self.address_mappings.get(&action) {
            self.osc_client.send(addr, action.value())?;
        }
        Ok(())
    }
}

---

Latency Budget

StageBudgetNotes
Gesture detection500μsFrom cc-gesture
Action scheduling100μsQueue insertion
Beat quantization50μsNext beat calculation
MIDI/OSC send200μsNetwork I/O
Audio callback5.3ms256 samples @ 48kHz
Total< 6msEnd-to-end

---

Configuration

rust
pub struct ConductorConfig {
    pub base_tempo: f32,        // 120.0
    pub tempo_range: (f32, f32), // (80.0, 160.0)
    pub tempo_smoothing: f32,   // 0.95
    pub quantize_mode: QuantizeMode, // Bar
    pub link_enabled: bool,     // true
}

pub struct AudioConfig {
    pub sample_rate: u32,       // 48000
    pub buffer_size: usize,     // 256
    pub channels: usize,        // 2
    pub output_device: String,
}

---

AUDIT FINDINGS

FindingSeverityStatusNotes
Lock-free queuesPassUsing crossbeam
Buffer pre-allocationPassAll buffers sized at init
Link integrationNoteOptionalFallback to local timing
MIDI latency⚠️MeasureMay vary by device
Tempo smoothingNoteTune0.95 might be too slow

---

Future Work

1. VST/AU hosting: Load audio plugins
2. Waveform analysis: Beat/key detection
3. Phrase detection: Automatic phrase boundaries
4. Crowd energy: External energy input

---

Further Reading

  • [03-ECHELON.md](03-ECHELON.md) - Echelon overview
  • [07-GESTURE_RECOGNITION.md](07-GESTURE_RECOGNITION.md) - Input gestures
  • [12-DEPLOYMENT.md](12-DEPLOYMENT.md) - Audio device setup

Promotion Decision

Promote into a technical note or architecture paper with implementation anchors.

Source Anchor

Comp-Core/docs/architecture/10-AUDIO_ENGINE.md

Detected Structure

Method · Evaluation · Architecture