Audio Engine
**Version**: 1.1.0 **Last Updated**: 2025-01-01 **Status**: Production **Parent**: [03-ECHELON.md](03-ECHELON.md) **Previous**: [07-GESTURE_RECOGNITION.md](07-GESTURE_RECOGNITION.md) **Crate**: `core/cc-echelon/crates/cc-conductor/` **Tests**: 31 passing
Full Public Reader
Audio Engine
Layer 5-7: Beat Scheduling and DSP
Version: 1.1.0
Last Updated: 2025-01-01
Status: Production
Parent: [03-ECHELON.md](03-ECHELON.md)
Previous: [07-GESTURE_RECOGNITION.md](07-GESTURE_RECOGNITION.md)
Crate: `core/cc-echelon/crates/cc-conductor/`
Tests: 31 passing
---
Overview
The audio engine provides beat-quantized scheduling, real-time DSP, and DJ software integration. It operates under strict latency constraints where every millisecond matters.
Latency Budget
| Stage | Budget | Notes |
|---|---|---|
| Gesture detection | 500μs | From cc-gesture |
| Action scheduling | 100μs | Queue insertion |
| Beat quantization | 50μs | Next beat calculation |
| MIDI/OSC send | 200μs | Network I/O |
| Audio callback | 5.3ms | 256 samples @ 48kHz |
| Total | < 6ms | End-to-end |
---
Architecture
flowchart TB
subgraph Input [Input]
Lexicon[Lexicon Signals]
Gesture[Gesture Events]
Link[Ableton Link]
end
subgraph Conductor [cc-conductor]
Tempo[Tempo Tracker]
Quantizer[Beat Quantizer]
Actions[Action Queue]
end
subgraph Scheduler [scheduler]
Timeline[Beat Timeline]
Pending[Pending Actions]
Fire[Fire Actions]
end
subgraph Engine [audio-engine]
DSP[DSP Chain]
Mixer[Mixer]
Output[Audio Output]
end
subgraph DJ [DJ Integration]
RB[Rekordbox]
Serato[Serato]
MIDI[MIDI/OSC]
end
Lexicon --> Tempo
Gesture --> Actions
Link --> Tempo
Tempo --> Quantizer --> Actions --> Pending
Pending --> Fire --> DSP --> Mixer --> Output
Fire --> MIDI --> RB
Fire --> MIDI --> Serato---
cc-conductor
Orchestrates motion-to-music translation:
pub struct Conductor {
config: ConductorConfig,
tempo: TempoTracker,
section: SectionState,
pattern_engine: PatternEngine,
action_queue: VecDeque<ScheduledAction>,
}
impl Conductor {
pub fn process(
&mut self,
lexicon: &Lexicon,
gesture: Option<&GestureEvent>,
dt: f32,
) -> Vec<Action> {
// Update tempo from lexicon
self.tempo.update(lexicon.energy, dt);
// Update section state
self.section = self.compute_section(lexicon);
// Generate pattern modifications
let pattern_changes = self.pattern_engine.process(lexicon, self.section);
// Process gestures into actions
let gesture_actions = gesture.map(|g| self.gesture_to_action(g));
// Collect and quantize actions
self.collect_actions(pattern_changes, gesture_actions)
}
}TempoTracker
Smoothly adapts tempo to motion energy:
pub struct TempoTracker {
base_bpm: f32, // 120.0
current_bpm: f32,
target_bpm: f32,
smoothing: f32, // 0.95 (high smoothing)
min_bpm: f32, // 80.0
max_bpm: f32, // 160.0
}
impl TempoTracker {
pub fn update(&mut self, energy: f32, dt: f32) {
// Map energy [0,1] to tempo range
self.target_bpm = self.min_bpm + energy * (self.max_bpm - self.min_bpm);
// Smooth transition
self.current_bpm = self.smoothing * self.current_bpm
+ (1.0 - self.smoothing) * self.target_bpm;
}
pub fn beat_duration(&self) -> f32 {
60.0 / self.current_bpm
}
}---
Beat Scheduler
Quantizes actions to the beat grid:
pub struct BeatScheduler {
timeline: BeatTimeline,
pending: Vec<ScheduledAction>,
quantize_mode: QuantizeMode,
}
pub enum QuantizeMode {
Off, // Fire immediately
Beat, // Quantize to beat
Bar, // Quantize to bar
Phrase, // Quantize to 4/8 bar phrase
}
pub struct ScheduledAction {
pub action: Action,
pub fire_at: BeatPosition,
pub priority: Priority,
}
impl BeatScheduler {
pub fn schedule(&mut self, action: Action, quantize: QuantizeMode) {
let fire_at = match quantize {
QuantizeMode::Off => self.timeline.now(),
QuantizeMode::Beat => self.timeline.next_beat(),
QuantizeMode::Bar => self.timeline.next_bar(),
QuantizeMode::Phrase => self.timeline.next_phrase(),
};
self.pending.push(ScheduledAction {
action,
fire_at,
priority: Priority::Normal,
});
}
pub fn tick(&mut self, now: BeatPosition) -> Vec<Action> {
let (ready, pending): (Vec<_>, Vec<_>) = self.pending
.drain(..)
.partition(|a| a.fire_at <= now);
self.pending = pending;
ready.into_iter().map(|a| a.action).collect()
}
}BeatTimeline
Integrates with Ableton Link for network sync:
pub struct BeatTimeline {
link: Option<AbletonLink>,
local_tempo: f32,
beat_phase: f32,
bar_phase: f32,
}
impl BeatTimeline {
pub fn update(&mut self, dt: f32) {
if let Some(ref link) = self.link {
// Sync from Link
let state = link.capture_audio_session_state();
self.beat_phase = state.beat_at_time(link.clock_micros()) % 1.0;
self.bar_phase = state.beat_at_time(link.clock_micros()) % 4.0 / 4.0;
} else {
// Local timing
self.beat_phase = (self.beat_phase + dt / self.beat_duration()) % 1.0;
self.bar_phase = (self.bar_phase + dt / (self.beat_duration() * 4.0)) % 1.0;
}
}
pub fn next_beat(&self) -> BeatPosition {
BeatPosition::from_phase(1.0 - self.beat_phase)
}
}---
Real-Time DSP
Hard real-time audio processing with zero-allocation callbacks:
pub struct AudioEngine {
sample_rate: u32, // 48000
buffer_size: usize, // 256
channels: Vec<Channel>,
master: MasterChannel,
}
impl AudioEngine {
/// Audio callback - MUST be lock-free, allocation-free
pub fn process(&mut self, output: &mut [f32]) {
// Clear output
for sample in output.iter_mut() {
*sample = 0.0;
}
// Mix channels
for channel in &mut self.channels {
if channel.is_active() {
channel.render_into(output);
}
}
// Master processing
self.master.process(output);
}
}Safety Rules
| Rule | Enforcement |
|---|---|
| No allocation | Pre-allocated buffers |
| No blocking | Lock-free queues |
| No syscalls | Avoid I/O in callback |
| Bounded time | Fixed buffer size |
---
DJ Integration
Action Types
pub enum Action {
// Transport
Play,
Pause,
CueSet,
CueGoto,
// Mixing
CrossfaderLeft(f32),
CrossfaderRight(f32),
EqLow(u8, f32), // (deck, value)
EqMid(u8, f32),
EqHigh(u8, f32),
// Effects
EffectOn(u8), // effect slot
EffectOff(u8),
EffectDry(u8, f32),
// Loops
LoopIn,
LoopOut,
LoopDouble,
LoopHalf,
// Navigation
JumpForward(f32), // beats
JumpBack(f32),
}Rekordbox Bridge
pub struct RekordboxBridge {
midi_out: MidiOutput,
key_mappings: HashMap<Action, MidiMessage>,
}
impl RekordboxBridge {
pub fn send(&mut self, action: Action) -> Result<()> {
if let Some(msg) = self.key_mappings.get(&action) {
self.midi_out.send(msg)?;
}
Ok(())
}
}Serato Bridge
pub struct SeratoBridge {
osc_client: OscClient,
address_mappings: HashMap<Action, String>,
}
impl SeratoBridge {
pub fn send(&mut self, action: Action) -> Result<()> {
if let Some(addr) = self.address_mappings.get(&action) {
self.osc_client.send(addr, action.value())?;
}
Ok(())
}
}---
Latency Budget
| Stage | Budget | Notes |
|---|---|---|
| Gesture detection | 500μs | From cc-gesture |
| Action scheduling | 100μs | Queue insertion |
| Beat quantization | 50μs | Next beat calculation |
| MIDI/OSC send | 200μs | Network I/O |
| Audio callback | 5.3ms | 256 samples @ 48kHz |
| Total | < 6ms | End-to-end |
---
Configuration
pub struct ConductorConfig {
pub base_tempo: f32, // 120.0
pub tempo_range: (f32, f32), // (80.0, 160.0)
pub tempo_smoothing: f32, // 0.95
pub quantize_mode: QuantizeMode, // Bar
pub link_enabled: bool, // true
}
pub struct AudioConfig {
pub sample_rate: u32, // 48000
pub buffer_size: usize, // 256
pub channels: usize, // 2
pub output_device: String,
}---
AUDIT FINDINGS
| Finding | Severity | Status | Notes |
|---|---|---|---|
| Lock-free queues | ✓ | Pass | Using crossbeam |
| Buffer pre-allocation | ✓ | Pass | All buffers sized at init |
| Link integration | Note | Optional | Fallback to local timing |
| MIDI latency | ⚠️ | Measure | May vary by device |
| Tempo smoothing | Note | Tune | 0.95 might be too slow |
---
Future Work
1. VST/AU hosting: Load audio plugins
2. Waveform analysis: Beat/key detection
3. Phrase detection: Automatic phrase boundaries
4. Crowd energy: External energy input
---
Further Reading
- [03-ECHELON.md](03-ECHELON.md) - Echelon overview
- [07-GESTURE_RECOGNITION.md](07-GESTURE_RECOGNITION.md) - Input gestures
- [12-DEPLOYMENT.md](12-DEPLOYMENT.md) - Audio device setup
Promotion Decision
Promote into a technical note or architecture paper with implementation anchors.
Source Anchor
Comp-Core/docs/architecture/10-AUDIO_ENGINE.md
Detected Structure
Method · Evaluation · Architecture