Stage 2: COMPOUND

VOICE PROCESSING PIPELINE:

Audio from LUME mic array (3x MEMS, beamformed)
     |
     v
Audio Preprocessor (Jetson CPU, <5ms)
  |-- Noise gate (RMS threshold, tuned for coffee shop 60-70 dB ambient)
  |-- Band-pass filter (200Hz - 4kHz, voice frequency isolation)
  |-- Automatic gain control (normalize to -3 dBFS)
  |-- Voice activity detection (energy-based, <1ms)
     |
     v
STT TIER 1 (Primary: Cloud Streaming)  <-- Path C adopted
  |-- Provider: Deepgram Nova-3 (streaming WebSocket)
  |-- Latency: ~300ms for partial transcripts
  |-- Accuracy: best-in-class for noisy environments
  |-- Cost: ~$2/month at 1000 orders
  |-- Requires: WiFi connectivity
     |
     | FALLBACK (WiFi down or >500ms latency detected)
     v
STT TIER 2 (Fallback: Local Whisper)  <-- Path A adopted
  |-- Engine: whisper.cpp with CUDA (base.en model, 200MB)
  |-- Latency: ~2-3 seconds for 10s audio
  |-- Accuracy: adequate for clear speech, degrades in noise
  |-- Requires: GPU time-slice (3ms/frame for ~5-10 frames)
  |-- Auto-activates when: WiFi unreachable OR cloud latency >500ms
     |
     v
NLU PIPELINE (Always Local, Jetson CPU, <50ms)  <-- Path C + A adopted
  |-- MenuAliasMatcher (ported from BWB VoiceNLUEngine)
  |-- ModifierDetector (ported from BWB VoiceNLUEngine)
  |-- QuantityExtractor (ported from BWB VoiceNLUEngine)
  |-- ConfidenceScorer (ported from BWB OrderParsingPipeline)
  |-- OrderResultMerger (ported from BWB)
  |
  |  NOTE: NO LLM for NLU. The BWB patterns are the domain expert.
  |  Qwen3-4B rejected for coffee NLU. The regex+fuzzy patterns
  |  in BWB's 80K lines of NLU code are faster, more accurate,
  |  and use zero GPU/memory for the coffee ordering domain.
     |
     v
Cart State Machine (ported from BWB, CPU)
  |-- CartCoordinator
  |-- ConfirmationCoordinator (auto-confirm at 0.8 confidence, 3s countdown)
  |-- SessionManager (120s timeout)
     |
     v
TTS: Piper (local, CPU, <100ms)  <-- Path A adopted
  |-- Pre-loaded voice model (~50MB)
  |-- Confirmation: "Large oat milk latte, $5.50. Anything else?"
  |-- Order ready: "Sarah, your latte is ready!"
  |-- Error: "Sorry, I didn't catch that. Could you say it again?"
     |
     v
LUME Speakers (stereo 3W) OR HDMI audio pass-through

ANALYTICS PIPELINE (continuous, ~0.5ms additional GPU per frame):

Existing Depth Pipeline:
  DepthReprojection.compute -> Point Cloud + Silhouette Mask
       |
       |  BRANCHING: same silhouette data serves both visual engine
       |  and analytics engine simultaneously
       v
  BodyDetector (Rust, CPU, <1ms)
    |-- Input: silhouette mask from depth compute shader
    |-- Connected component labeling on binary mask
    |-- Filter: minimum blob size (0.3m x 0.5m at depth distance)
    |-- Output: list of body bounding boxes + centroids
       |
       v
  CentroidTracker (Rust, CPU, <0.5ms per frame)
    |-- Kalman filter per tracked centroid (predict + update)
    |-- Track ID assignment via Hungarian algorithm (scipy equivalent in Rust)
    |-- Occlusion handling: predict through 2-second disappearance
    |-- Track birth: new centroid unmatched for 0.5s
    |-- Track death: centroid unmatched for 3s
    |-- Output: list of tracks with (id, position, velocity, age, zone)
       |
       v
  ZoneClassifier (Rust, CPU, trivial)
    |-- Configured zones (admin setup via companion app):
    |     QUEUE: rectangle covering the waiting area
    |     SERVICE: rectangle at the counter
    |     ENTRANCE: rectangle at the door
    |     EXIT: rectangle at the exit (may overlap entrance)
    |     SEATING: rectangle for dine-in area
    |-- Each track.position -> zone assignment
    |-- Zone transitions logged: track 17 moved from QUEUE to SERVICE at t=142s
       |
       v
  MetricsAggregator (Rust, CPU, runs every 5 seconds)
    |-- REAL-TIME METRICS:
    |     current_occupancy: count of active tracks
    |     queue_length: count of tracks in QUEUE zone
    |     estimated_wait: queue_length * avg_service_time
    |     bodies_in_service: count of tracks in SERVICE zone
    |
    |-- ROLLING WINDOW METRICS (5min / 15min / 1hr / 1day):
    |     avg_wait_time: mean dwell in QUEUE zone before SERVICE
    |     avg_service_time: mean dwell in SERVICE zone
    |     throughput: count of QUEUE->SERVICE transitions per hour
    |     conversion_rate: (reached SERVICE) / (entered ENTRANCE)
    |     abandonment_rate: (ENTRANCE->EXIT without SERVICE)
    |     peak_occupancy: 95th percentile of occupancy
    |
    |-- HEATMAP (updated every 60 seconds):
    |     2D grid (20x20) of time-weighted centroid density
    |     Normalized for display on web dashboard
    |
    |-- FLOW PATHS (updated every 5 minutes):
    |     Most common zone transition sequences
    |     e.g., ENTRANCE -> QUEUE -> SERVICE -> SEATING -> EXIT (72%)
    |          ENTRANCE -> QUEUE -> EXIT (8% = abandonment)
       |
       v
  DataExporter
    |-- Local: SQLite on NVMe (raw track events, 7-day retention)
    |-- Cloud: Supabase analytics tables (aggregated metrics, 5s intervals)
    |-- Dashboard: WebSocket push to web dashboard (real-time)
    |-- API: REST endpoint on LUME for direct queries

PAYMENT OPTIONS (shop owner chooses during setup):

OPTION 1: Square Terminal ($299, self-contained)
  |-- LUME creates order -> REST API to Square Terminal
  |-- Terminal displays order on its own screen
  |-- Customer taps/swipes card on Terminal
  |-- Terminal processes payment via Square
  |-- Terminal sends confirmation callback to LUME
  |-- LUME announces "Payment confirmed!" + visual celebration
  |-- Pros: Terminal has receipt printer, own screen, established
  |-- Cons: $299 additional hardware, separate device management

OPTION 2: Stripe Terminal S700 ($349, smart reader)
  |-- Same REST API flow as Square
  |-- S700 has own screen + receipt display
  |-- Server-driven integration (Stripe API, no Linux SDK needed)
  |-- Pros: Stripe ecosystem, developer-friendly API
  |-- Cons: $349, requires Stripe account setup

OPTION 3: Pay-at-Counter (zero additional hardware)
  |-- LUME takes voice order only
  |-- Order displays on barista POS (existing system)
  |-- Customer pays at counter via existing POS
  |-- LUME is notified of payment via WebSocket from POS
  |-- Pros: zero additional cost, works with any POS
  |-- Cons: customer walks to counter to pay (no queue-time payment)

OPTION 4: NFC Tap-to-Pay on LUME (future, $15 NFC reader component)
  |-- Small NFC reader module added to LUME hardware (BOM: ~$15)
  |-- Apple Pay / Google Pay tap directly on LUME surface
  |-- Server-driven Stripe Terminal API processes the tap
  |-- Pros: one-device experience, magical
  |-- Cons: NFC reader adds BOM, Stripe server-driven integration
  |-- Timeline: V2 hardware only (V1 uses external reader or counter pay)

DEFAULT RECOMMENDATION for V1: Option 3 (pay-at-counter)
  Reason: zero additional hardware, zero integration complexity,
  shop owner keeps their existing payment flow. LUME adds value
  by taking the ORDER, not by processing the PAYMENT.

  Option 1 (Square Terminal) available for shops wanting full
  self-service flow. Rest API integration is ~300 lines of Rust.

CONTENT FLYWHEEL:

1. ATTRACT: Customer enters queue, LUME detects body
   - Visual effects activate around their silhouette
   - Other waiting customers see the interaction and engage

2. INTERACT: Customer moves, waves, dances
   - Fluid particles react to their motion
   - Duration: average 45-120 seconds during queue wait
   - LUME continuously buffers last 30 seconds (circular buffer)

3. CAPTURE: Auto-triggered when:
   - Body has been in QUEUE zone for >15 seconds (engaged, not just passing)
   - Motion energy exceeds threshold (they're actually interacting)
   - LUME captures 15-second clip: camera feed + visual overlay + audio
   - Clip stored on NVMe, composited in <5 seconds (NVENC)

4. OFFER: QR code appears on LUME display
   - "Get your clip!" with QR code linking to:
     https://lume.dance/clip/{venue_slug}/{clip_id}
   - Web page: preview + download (no app required)
   - Clip metadata: venue name, location, date, visual preset used
   - Optional: venue logo watermark in corner

5. TRANSFER: Customer scans QR with phone
   - Web page loads clip preview (HLS streaming)
   - One-tap download to camera roll
   - One-tap share to TikTok / Instagram / Stories
   - Clip includes: venue location tag, LUME credit, royalty-free audio

6. VIRAL: Posted content drives awareness
   - Location tag -> friends discover the venue
   - Visual uniqueness -> "what is that?" comments
   - LUME credit -> curious viewers search for LUME
   - Venue gets free social media marketing

ANALYTICS (for venue owner dashboard):
  - Clips captured today: N
  - QR scans (engagement): N (% of clips)
  - Downloads: N (% of scans)
  - Estimated social shares: N (based on download-to-post ratio)
  - Content impressions (if venue connects social accounts): N

VISUAL TRANSITION:

Normal Mode:
  [Full-screen fluid particles, depth-reactive, beautiful]

Voice Ordering Active:
  [Particles smoothly compress to edges]
  [Center area clears with glass-frost effect]
  [Order items appear as glowing text cards floating in particle field]

  +---------------------------------------------+
  |  :::particles::: Large Oat Milk Latte $5.50  |
  |  :::particles:::  + Extra shot        $0.75  |
  |  :::particles:::                             |
  |  :::particles::: Total: $6.25                |
  |  :::particles:::                             |
  |  :::particles::: "Anything else?"            |
  +---------------------------------------------+
  [Customer's silhouette still visible behind semi-transparent cards]
  [Particles still react to movement, just displaced]

Order Confirmed:
  [Text cards dissolve into celebration particle burst]
  [Confetti-like effect using order's color theme]
  [Return to full visual mode]
  [Ticket number displayed briefly: "#42"]

ORDER ROUTING:

LUME (order confirmed)
     |
     v
OrderRouter (Rust, ~100 lines)
     |
     |-- Route 1: LUME's own HDMI output
     |     Kitchen mode: split screen, orders on right 30%
     |     Small shops where LUME display IS the kitchen display
     |
     |-- Route 2: WebSocket to companion kitchen screen
     |     Standard: any device with a browser (tablet, old phone, TV)
     |     URL: http://lume-local:8080/kitchen
     |     Shows: order queue, order details, status buttons
     |     Barista taps "started" / "ready" on the web page
     |     Status update -> LUME announces "Order ready!"
     |
     |-- Route 3: WebSocket to BWB_POS kitchen display
     |     Existing KitchenDisplayView.swift on a companion iPad
     |     Uses same Supabase realtime subscriptions
     |     Zero new code on the iPad side
     |
     |-- Route 4: API push to external POS (Square, Toast)
     |     REST API integration with venue's existing POS
     |     Order appears in their existing kitchen display
     |     Status callbacks via webhooks

DEFAULT: Route 2 (browser-based kitchen display)
  Reason: works on any device, no app to install,
  responsive web UI that looks like a KDS.
  Built as part of LUME's embedded web server.

LUME serves a web app at http://lume-local:8080/kitchen

Features:
  - Dark theme (kitchen-safe, high contrast)
  - Order cards in grid layout (similar to BWB KitchenDisplayView)
  - Color-coded: new (blue), in-progress (orange), ready (green)
  - Tap to change status
  - Timer per order (elapsed since placed)
  - Sound alert for new orders
  - Auto-print via connected receipt printer (ESC/POS over USB)
  - Works offline (orders cached, syncs when connection restored)

Tech: Rust Axum web server + htmx + minimal CSS
  Total: ~500 lines Rust + ~200 lines HTML/CSS/JS

LUME COMMERCE SOFTWARE STACK (all running on Jetson Orin Nano Super):

+==============================================================+
|                        LUME OS (L4T Ubuntu)                   |
|                                                               |
|  +----------------------------------------------------------+|
|  |              Visual Engine (Unity URP)                     ||
|  |  DepthReprojection -> OpticalFlow -> FluidSim -> VFX     ||
|  |  ColorPalette -> Content Compositor -> NVENC recorder     ||
|  |  + ORDER OVERLAY RENDERER (new: text cards in particles) ||
|  +----------------------------------------------------------+|
|                                                               |
|  +----------------------------------------------------------+|
|  |              Commerce Engine (Rust)                        ||
|  |                                                            ||
|  |  VoiceOrderingOrchestrator (ported from BWB)              ||
|  |    |-- AudioPreprocessor (noise gate + AGC + bandpass)    ||
|  |    |-- STTBridge                                          ||
|  |    |     |-- Primary: DeepgramClient (WebSocket streaming)||
|  |    |     |-- Fallback: WhisperBridge (whisper.cpp + CUDA) ||
|  |    |-- NLUPipeline (ported from BWB, Rust)                ||
|  |    |     |-- MenuAliasMatcher                             ||
|  |    |     |-- ModifierDetector                             ||
|  |    |     |-- QuantityExtractor                            ||
|  |    |     |-- ConfidenceScorer                             ||
|  |    |-- CartCoordinator (ported from BWB)                  ||
|  |    |-- ConfirmationCoordinator (ported from BWB)          ||
|  |    |-- SessionManager (120s timeout)                      ||
|  |    |-- PiperTTS (local voice synthesis)                   ||
|  |    |-- PaymentBridge (REST to Square/Stripe Terminal)     ||
|  |    |-- KitchenRouter (WebSocket to kitchen displays)      ||
|  |                                                            ||
|  |  QueueAnalyticsEngine (Rust)                              ||
|  |    |-- BodyDetector (from depth silhouette mask)          ||
|  |    |-- CentroidTracker (Kalman filter + Hungarian)        ||
|  |    |-- ZoneClassifier (configurable regions)              ||
|  |    |-- MetricsAggregator (rolling windows)                ||
|  |    |-- HeatmapGenerator                                   ||
|  |    |-- DataExporter (SQLite local + Supabase cloud)       ||
|  |                                                            ||
|  |  ContentFlywheel (Rust)                                   ||
|  |    |-- InteractionDetector (motion energy in QUEUE zone)  ||
|  |    |-- ClipCapture (15s, triggered by engagement)         ||
|  |    |-- QRCodeGenerator (dynamic, per-clip URL)            ||
|  |    |-- ContentServer (http://lume-local:8080/clips/)      ||
|  |    |-- ContentUploader (to Supabase/CDN for web access)   ||
|  |                                                            ||
|  +----------------------------------------------------------+|
|                                                               |
|  +----------------------------------------------------------+|
|  |              Device Manager (Rust, systemd)                ||
|  |  Boot sequence, OTA updates, WiFi AP, health monitoring   ||
|  |  + Menu Config Manager (JSON, OTA-updatable from Supabase)||
|  |  + Zone Config Manager (admin setup via companion app)    ||
|  |  + Venue Config (name, branding, payment option, kitchen) ||
|  +----------------------------------------------------------+|
|                                                               |
|  +----------------------------------------------------------+|
|  |              Echelon Audio Engine (Rust, existing)          ||
|  |  Audio synthesis (GENERATE mode) / mic FFT (LISTEN mode)  ||
|  +----------------------------------------------------------+|
|                                                               |
|  +----------------------------------------------------------+|
|  |              Web Server (Rust Axum)                        ||
|  |  :8080/kitchen  -- Kitchen Display (htmx)                 ||
|  |  :8080/dashboard -- Analytics Dashboard (htmx)            ||
|  |  :8080/clips/    -- Content clip web pages                ||
|  |  :8080/api/      -- REST API for companion app + external ||
|  |  :8080/ws        -- WebSocket for real-time data          ||
|  +----------------------------------------------------------+|
+==============================================================+

lume-commerce/
  src/
    voice/
      mod.rs                    -- Voice module root
      orchestrator.rs           -- VoiceOrderingOrchestrator (ported from BWB, ~400 lines)
      audio_preprocessor.rs     -- Noise gate + bandpass + AGC (~150 lines)
      stt_bridge.rs             -- STT tier selection + Deepgram + Whisper (~300 lines)
      deepgram_client.rs        -- WebSocket streaming to Deepgram (~200 lines)
      whisper_bridge.rs         -- FFI to whisper.cpp library (~150 lines)
      piper_bridge.rs           -- FFI to Piper TTS (~100 lines)
    nlu/
      mod.rs                    -- NLU module root
      menu_matcher.rs           -- MenuAliasMatcher (ported from BWB, ~300 lines)
      modifier_detector.rs      -- ModifierDetector (ported from BWB, ~200 lines)
      quantity_extractor.rs     -- QuantityExtractor (ported from BWB, ~100 lines)
      confidence_scorer.rs      -- ConfidenceScorer (ported from BWB, ~150 lines)
      order_merger.rs           -- OrderResultMerger (ported from BWB, ~100 lines)
      vocabulary.rs             -- Menu vocabulary loader (JSON config, ~100 lines)
    cart/
      mod.rs                    -- Cart module root
      coordinator.rs            -- CartCoordinator (ported from BWB, ~200 lines)
      confirmation.rs           -- ConfirmationCoordinator (ported, ~150 lines)
      session.rs                -- SessionManager (ported, ~100 lines)
      models.rs                 -- Order, MenuItem, Customization models (~200 lines)
    payment/
      mod.rs                    -- Payment module root
      bridge.rs                 -- PaymentBridge (REST to Square/Stripe, ~200 lines)
      square_terminal.rs        -- Square Terminal API client (~150 lines)
      stripe_terminal.rs        -- Stripe Terminal server-driven client (~150 lines)
    analytics/
      mod.rs                    -- Analytics module root
      body_detector.rs          -- BodyDetector from depth mask (~150 lines)
      centroid_tracker.rs       -- Kalman + Hungarian tracking (~300 lines)
      zone_classifier.rs        -- Zone assignment (~100 lines)
      metrics.rs                -- MetricsAggregator (~250 lines)
      heatmap.rs                -- 2D heatmap generator (~100 lines)
      exporter.rs               -- SQLite + Supabase + WebSocket export (~200 lines)
    content/
      mod.rs                    -- Content module root
      interaction_detector.rs   -- Motion energy threshold for clip trigger (~100 lines)
      clip_capture.rs           -- 15s clip management (~150 lines)
      qr_generator.rs           -- QR code generation (~50 lines)
      content_server.rs         -- HTTP endpoints for clip web pages (~150 lines)
    kitchen/
      mod.rs                    -- Kitchen module root
      router.rs                 -- Order routing to kitchen displays (~150 lines)
      web_display.rs            -- Embedded kitchen display web app (~200 lines)
    web/
      mod.rs                    -- Web server root
      server.rs                 -- Axum server setup (~200 lines)
      api.rs                    -- REST API endpoints (~300 lines)
      websocket.rs              -- WebSocket handlers (~200 lines)
    config/
      mod.rs                    -- Configuration module
      menu.rs                   -- Menu config loader (~100 lines)
      zones.rs                  -- Zone config loader (~80 lines)
      venue.rs                  -- Venue branding/settings (~80 lines)
    main.rs                     -- Commerce engine entry point (~100 lines)

TOTAL ESTIMATED: ~5,500 lines of Rust

PHASE 1: V0.5 Pilot (Week 1-2)
  - Deploy LUME with visual engine + queue analytics ONLY
  - No voice ordering yet (proving entertainment + analytics value)
  - Companion iPad runs BWB_Kiosk for ordering (Path B for V0.5)
  - WebSocket relay: LUME sends queue data -> iPad displays wait time
  - Measure: customer engagement time, QR scans, social shares

PHASE 2: Voice Ordering Alpha (Week 3-4)
  - Enable Deepgram STT on LUME (cloud streaming)
  - Port BWB NLU patterns to Rust (coffee menu only)
  - Voice ordering: customer speaks -> LUME processes -> order to kitchen
  - Payment: pay at counter (existing POS)
  - Measure: order accuracy, processing time, customer satisfaction

PHASE 3: Full Commerce (Week 5-6)
  - Add content flywheel (auto-capture, QR, web clip download)
  - Add order overlay in visual experience
  - Kitchen display on barista's tablet (web-based)
  - Analytics dashboard live on Supabase/Nexus
  - Measure: everything. Kill criteria at Day 14.

KILL CRITERIA:
  Day 14: If voice ordering accuracy <80%, fall back to iPad-only ordering
  Day 30: If content QR scan rate <10% of queue customers, simplify content flow
  Day 42: If barista satisfaction <7/10, rethink kitchen routing UX

TARGET: Multi-location coffee chains (10-200 locations)

HOOK: "Your queue is costing you customers. 8% of people who enter your
       store leave before ordering (industry average). LUME turns the
       queue into entertainment that keeps customers engaged AND creates
       free social media marketing."

DEMO: Live LUME with depth-reactive visuals in a meeting room.
      Prospect interacts. Captures a clip. Gets the QR. Downloads.
      "This happens with every customer in your queue."

DATA: Show analytics dashboard with mock data from pilot location.
      Queue length over time. Wait time trends. Content shares.
      A/B comparison: locations with LUME vs without.

ASK: "Pilot in 3 locations for 90 days. $1,299/device + $149/month.
      We install, configure, and support. If metrics don't justify
      continuing, we buy back the hardware."

EXPANSION: After successful pilot -> fleet rollout.
      Volume pricing: $999/device at 20+ units.
      Enterprise tier: custom pricing per location.
      Cross-location benchmarking dashboard included.

TIMELINE:
  Month 1: Initial meeting + demo
  Month 2: Pilot agreement, 3 locations installed
  Month 3-4: Pilot period (90 days)
  Month 5: Review metrics, negotiate fleet deal
  Month 6: Fleet rollout begins