Gesture Control System - Production Deployment Guide

Gesture Control System - Production Deployment Guide

Overview

This guide covers deploying the production-grade gesture control system with enterprise features including auto-recovery, monitoring, and performance optimization.

---

System Architecture

Production Components

#### Phase 1: Data Acquisition
- `sensor_logger_bridge_production.py` - Sensor streaming with auto-reconnection
- `gemini_video_analyzer_production.py` - Video analysis with adaptive FPS

#### Phase 2: Training & Recognition
- `gesture_database_production.py` - Template storage with versioning
- `gesture_recorder_production.py` - Recording with auto-save
- `gesture_recognizer_production.py` - Recognition with caching
- `training_ui_production.py` - Interactive training interface

---

Production Features

1. Reliability & Recovery

#### Auto-Reconnection
- Exponential backoff: 1s → 2s → 4s → 8s → ... → 60s max
- Connection state tracking: DISCONNECTED, CONNECTING, CONNECTED, RECONNECTING, FAILED
- Automatic recovery from network interruptions

#### Data Persistence
- Transaction-based saves with atomic writes
- Automatic backups every 5 minutes
- Session recovery after crashes
- Database corruption detection and rollback

2. Performance Optimization

#### Caching
- Template caching with 5-minute TTL
- Cache hit rate monitoring
- Automatic cache invalidation

#### Adaptive Rate Control
- Video FPS: Adapts 1-10 FPS based on API latency
- Frame quality filtering: Blur + brightness checks
- Circular buffering: Memory-efficient (max 1000 readings)

#### Batch Processing
- Parallel template comparison
- Feature extraction optimization
- Lazy loading of templates

3. Monitoring & Logging

#### Metrics Tracked
- Recognition success rate
- Average processing latency
- Cache hit/miss rates
- Database health status
- Recording quality scores

Logging Levels

logging.INFO   # Normal operations
logging.WARNING # Recoverable issues
logging.ERROR   # Critical failures
logging.DEBUG   # Detailed diagnostics

4. Data Quality

#### Validation
- Sensor reading validation: Physical plausibility checks
- Feature validation: Range and type checks
- Quality scoring: 0-1 based on completeness, consistency, magnitude

#### Outlier Detection
- Statistical outliers: IQR method (1.5 × IQR)
- 3-sigma rule: Values beyond 3 std from mean
- Automatic outlier removal during training

#### Cross-Validation
- Leave-one-out validation for accuracy measurement
- Consistency scoring based on feature variance
- Template versioning for iterative improvement

---

Deployment Steps

1. Environment Setup

# Install dependencies
pip install numpy websockets opencv-python google-genai

# Create directories
mkdir -p gesture_database/samples
mkdir -p gesture_database/backups
mkdir -p recording_sessions

2. Phone Configuration

Install Sensor Logger app on phone:
- iOS: [Sensor Logger](https://apps.apple.com/app/sensor-logger/id1531582925)
- Android: [Sensor Logger](https://play.google.com/store/apps/details?id=com.kelvin.sensorapp)

Configure WebSocket streaming:

Protocol: WebSocket
Host: <your-computer-ip>
Port: 8765

3. Gemini API Setup

Get API key from [Google AI Studio](https://aistudio.google.com/):

# Set environment variable
export GEMINI_API_KEY="your-api-key-here"

# Or in .env file
echo "GEMINI_API_KEY=your-api-key-here" >> .env

4. Run Training System

import asyncio
from dj_agent.gesture_control import (
    SensorLoggerBridge,
    GeminiVideoAnalyzer,
    TrainingUI,
)

async def main():
    # Initialize sensor bridge
    sensor_bridge = SensorLoggerBridge(
        host="[ip]",
        port=8765,
        enable_calibration=True,
    )

    # Initialize video analyzer
    video_analyzer = GeminiVideoAnalyzer(
        [sensitive field redacted]),
        target_fps=5,
        enable_quality_filter=True,
    )

    # Start services
    await sensor_bridge.start_streaming()
    await video_analyzer.start()

    # Create and run training UI
    ui = TrainingUI(
        sensor_bridge=sensor_bridge,
        session_file="./training_session.json",
    )

    await ui.run()

    # Cleanup
    await video_analyzer.stop()
    await sensor_bridge.stop_streaming()

if __name__ == '__main__':
    asyncio.run(main())

---

Configuration

Database Settings

database = GestureDatabase(
    db_path="./gesture_database",
)

# Configuration in gesture_database_production.py
MAX_BACKUP_COUNT = 10          # Keep last 10 backups
BACKUP_RETENTION_DAYS = 30     # Delete backups older than 30 days
AUTO_BACKUP_INTERVAL = 300     # Auto-backup every 5 minutes (seconds)

Recognizer Settings

recognizer = GestureRecognizer(
    database=database,
    confidence_threshold=0.7,    # Minimum confidence (0-1)
    enable_caching=True,          # Enable template caching
    enable_calibration=True,      # Enable confidence calibration
)

# Adjust feature weights
recognizer.feature_weights['gyro_direction'] = 2.0  # Critical
recognizer.feature_weights['accel_peak_x'] = 1.5     # High importance
recognizer.feature_weights['motion_smoothness'] = 0.8  # Low importance

Recorder Settings

recorder = GestureRecorder(
    sensor_bridge=sensor_bridge,
    video_analyzer=video_analyzer,
    database=database,
    auto_save=True,               # Enable auto-save
    session_dir="./recording_sessions",
)

# Configuration in gesture_recorder_production.py
MAX_BUFFER_SIZE = 1000          # Max sensor readings to buffer
MIN_SENSOR_READINGS = 10        # Minimum for valid recording
SENSOR_QUALITY_THRESHOLD = 0.8  # Minimum quality score (0-1)

Sensor Bridge Settings

sensor_bridge = SensorLoggerBridge(
    host="[ip]",
    port=8765,
    enable_calibration=True,
    calibration_samples=100,      # Samples for calibration
    reading_buffer_size=100,      # Circular buffer size
)

# Configuration in sensor_logger_bridge_production.py
RECONNECT_DELAY = 1.0           # Initial reconnect delay (seconds)
MAX_RECONNECT_DELAY = 60.0      # Max reconnect delay (seconds)
CONNECTION_TIMEOUT = 5.0        # Connection timeout (seconds)

Video Analyzer Settings

video_analyzer = GeminiVideoAnalyzer(
    [sensitive field redacted],
    target_fps=5,                 # Target frame rate
    min_fps=1,                    # Minimum FPS
    max_fps=10,                   # Maximum FPS
    enable_quality_filter=True,
    quality_threshold=0.6,        # Frame quality threshold
)

# Configuration in gemini_video_analyzer_production.py
FRAME_QUALITY_THRESHOLD = 0.6   # Minimum frame quality
MAX_RETRY_ATTEMPTS = 3          # API retry attempts
RETRY_DELAY = 1.0               # Retry delay (seconds)

---

Monitoring & Maintenance

Performance Dashboard

Access via Training UI → Option 5:

PERFORMANCE DASHBOARD
══════════════════════════════════════════════════════════════════════

🏥 Database Health:
  State: healthy
  Backups: 5

🎯 Recognition Performance:
  Success rate: 92%
  Avg latency: 23.4ms
  Cache hit rate: 87%

📹 Recording Performance:
  Success rate: 95%
  Avg quality: 89%

Log Monitoring

# View logs in real-time
tail -f gesture_control.log

# Filter by level
grep "ERROR" gesture_control.log
grep "WARNING" gesture_control.log

Database Backup Management

Backups are created automatically in `gesture_database/backups/`:

gesture_database/
├── backups/
│   ├── backup_20250122_140000/
│   ├── backup_20250122_145000/
│   └── backup_20250122_150000/

Manual backup:

database._create_timestamped_backup()

Restore from backup:

# Database auto-recovers, but you can manually restore:
cp gesture_database/backups/backup_YYYYMMDD_HHMMSS/templates.json \
   gesture_database/templates.json

Template Export/Import

Export template:

database.export_template("swipe_right", "./swipe_right_template.json")

Import template:

database.import_template("./swipe_right_template.json")

---

Troubleshooting

Connection Issues

Symptom: Sensor bridge not connecting

Solutions:
1. Check phone and computer are on same network
2. Verify IP address in phone app settings
3. Check firewall allows port 8765
4. Review logs: `grep "Connection" gesture_control.log`

Symptom: Frequent disconnections

Solutions:
1. Enable auto-reconnection (default)
2. Check network stability
3. Reduce sensor streaming rate in phone app

Recognition Issues

Symptom: Low recognition accuracy (<70

Solutions:
1. Record more training samples (15+ recommended)
2. Remove outlier samples: check database stats
3. Use practice mode to identify weak features
4. Lower confidence threshold temporarily

Symptom: High latency (>100ms)

Solutions:
1. Enable template caching (default)
2. Check cache hit rate in performance dashboard
3. Clear cache if templates updated: `recognizer.clear_cache()`

Database Issues

Symptom: Database corruption detected

Solutions:
1. Auto-recovery will attempt restore from backup
2. Check `gesture_database/backups/` for valid backups
3. Review logs: `grep "RECOVERING" gesture_control.log`

Symptom: Slow database saves

Solutions:
1. Check backup count: max 10 by default
2. Cleanup old backups manually if needed
3. Reduce `AUTO_BACKUP_INTERVAL` if too frequent

Video Analysis Issues

Symptom: Low frame quality warnings

Solutions:
1. Improve lighting conditions
2. Reduce motion blur (slower movements)
3. Lower quality threshold: `quality_threshold=0.5`

Symptom: High API latency

Solutions:
1. Adaptive FPS will auto-adjust (1-10 FPS)
2. Check internet connection
3. Review API quota limits

---

Best Practices

Training

1. Record in batches: 5 samples, break, repeat
2. Vary slightly: Don't be robotic, but keep core motion consistent
3. Practice mode: Achieve >85
4. Cross-validate: Run with `run_cross_validation=True`

Deployment

1. Monitor metrics: Check performance dashboard regularly
2. Review logs: Daily check for errors/warnings
3. Backup database: Export important templates
4. Version templates: Track template versions for rollback

Optimization

1. Cache hit rate: Aim for >80
2. Recognition latency: Target <50ms
3. Recording quality: Maintain >80
4. Database backups: Keep last 10, cleanup monthly

---

Performance Benchmarks

Expected Performance

Scaling Limits

---

Security Considerations

Data Privacy

• Local-first: All gesture data stored locally
• No cloud sync: Templates stay on device
• API key: Store in environment variable, never commit

Network Security

• WebSocket: Unencrypted by default (local network only)
• Firewall: Restrict port 8765 to local network
• Authentication: None (trusted local network assumed)

---

Upgrade Path

From Prototype to Production

1. Replace imports:

   # Before
   from dj_agent.gesture_control.trainer.gesture_database import GestureDatabase

   # After (automatic with __init__.py)
   from dj_agent.gesture_control import GestureDatabase

2. Update configuration: Add production settings
3. Migrate data: Use export/import for existing templates
4. Test thoroughly: Verify auto-recovery and reconnection

Future Enhancements

Potential improvements:
- Multi-user support
- Cloud sync for templates
- Real-time gesture streaming
- Advanced ML models (beyond template matching)
- Mobile app for training UI

---

Support & Resources

### Documentation
- [Training System Guide](./TRAINING_SYSTEM_GUIDE.md)
- [Gesture Control Architecture](./GESTURE_CONTROL_ARCHITECTURE.md)
- [Multimodal Creative Guide](./MULTIMODAL_CREATIVE_GUIDE.md)

### Code Examples
- `gesture_database_production.py` - See `main()` function
- `gesture_recognizer_production.py` - See `main()` function
- `training_ui_production.py` - See `main()` function

### Logging
All components use Python `logging` module:

import logging
logging.basicConfig(level=logging.INFO)

---

Production-Grade Gesture Control System
Version 1.0 - Enterprise Deployment
Author: Computational Choreography