Djoko Series Dataset Architecture

Djoko Series Dataset Architecture

🎯 Overview

The Djoko Series Dataset Creation System processes Djoko episodes into high-quality training data for:
- Bambara ASR (Automatic Speech Recognition)
- Bambara ↔ English Translation
- Bambara ↔ French Translation (future)
- Multimodal Language Learning

🏗️ Architecture Components

### 1. Voice Activity Detection (VAD)
Purpose: Remove silence and extract only speech segments

Features:
- Energy-based detection
- Spectral analysis for speech quality
- Adaptive thresholding
- Minimum segment duration filtering (0.5s+)
- Merge nearby segments (<0.3s gaps)
- Quality confidence scoring

Input: Raw Djoko episode audio
Output: List of speech segments with timestamps

### 2. Episode Processor
Purpose: Orchestrate the complete processing pipeline

Pipeline Steps:
1. Load Episode → Validate and prepare audio
2. Voice Activity Detection → Extract speech segments
3. Save Audio Segments → Individual WAV files per segment
4. ASR Transcription → Bambara text from audio
5. Translation → English (and French) translations
6. Generate Metadata → Complete episode information
7. Save Results → Structured training manifests

### 3. Training Data Generation
Purpose: Create training-ready datasets in multiple formats

Output Formats:
- ASR Training Manifest (NeMo JSONL format)
- Translation Pairs (JSON with Bambara ↔ English)
- Episode Metadata (Complete processing information)
- Quality Reports (Validation and statistics)

📊 Data Structure

Episode Structure

djoko_dataset/
├── episodes/
│   ├── episode_001/
│   │   ├── segments/
│   │   │   ├── djoko_ep001_segment_001_0.5s-3.2s.wav
│   │   │   ├── djoko_ep001_segment_002_4.1s-7.8s.wav
│   │   │   └── ...
│   │   ├── episode_metadata.json
│   │   ├── training_manifest.json
│   │   ├── asr_training_manifest.jsonl
│   │   └── translation_pairs.json
│   └── episode_002/
│       └── ...
├── manifests/
│   └── batch_processing_results.json
└── djoko_dataset_manifest.json

Training Segment Format

{
  "segment_id": "djoko_ep674_seg001",
  "episode_id": "djoko_ep674",
  "start_time": 0.5,
  "end_time": 3.2,
  "duration": 2.7,
  "audio_path": "djoko_dataset/episodes/episode_674/segments/djoko_ep674_segment_001_0.5s-3.2s.wav",
  "bambara_text": "bɛna tɔ do n'i yi finisi ba la",
  "english_text": "we will go there and finish the work",
  "french_text": null,
  "confidence_score": 0.85,
  "quality_metrics": {...}
}

ASR Training Manifest (NeMo Format)

{"audio_filepath": "path/to/segment.wav", "text": "bambara transcription", "duration": 2.7}
{"audio_filepath": "path/to/segment.wav", "text": "bambara transcription", "duration": 3.1}

Translation Training Pairs

[
  {
    "bambara": "bɛna tɔ do n'i yi finisi ba la",
    "english": "we will go there and finish the work",
    "segment_id": "djoko_ep674_seg001",
    "duration": 2.7
  }
]

🔄 Processing Pipeline

Single Episode Processing

processor = DjokoEpisodeProcessor()
results = processor.process_episode(
    audio_path="djoko_episode_674.wav",
    episode_number=674,
    episode_title="Djoko Episode 674"
)

Batch Processing

episodes_config = [
    {"audio_path": "episode_001.wav", "episode_number": 1},
    {"audio_path": "episode_002.wav", "episode_number": 2},
    # ... more episodes
]
results = processor.process_multiple_episodes(episodes_config)

🎯 Training Use Cases

### 1. Bambara ASR Training
Data: Audio segments + Bambara transcriptions
Format: NeMo JSONL manifest
Use: Train/fine-tune Bambara speech recognition models

### 2. Bambara ↔ English Translation
Data: Bambara text + English translations
Format: JSON pairs
Use: Train bidirectional translation models

### 3. Bambara ↔ French Translation
Data: Bambara text + French translations (future)
Format: JSON pairs
Use: Expand to French language support

### 4. Multimodal Learning
Data: Audio + Text + Metadata
Format: Complete training manifests
Use: Advanced language understanding models

📈 Quality Assurance

### Voice Activity Detection Quality
- Energy thresholding with adaptive adjustment
- Spectral analysis for speech vs. noise detection
- Confidence scoring for each segment
- Duration filtering to ensure meaningful segments

### ASR Quality Validation
- Confidence scores from ASR models
- Text length validation (reasonable transcription length)
- Character set validation (proper Bambara characters)

### Translation Quality
- Length ratio checks (reasonable translation lengths)
- Language detection validation
- Confidence scoring from translation models

🚀 Scalability Features

### Batch Processing
- Process multiple episodes in sequence
- Automatic error handling and recovery
- Progress tracking and reporting
- Resumable processing for large datasets

### Extensibility
- Modular design for easy component replacement
- Plugin architecture for new languages
- Configurable parameters for different use cases
- API-ready for integration with other systems

📊 Expected Output Statistics

For a typical Djoko episode (14+ minutes):
- Original Duration: ~860 seconds
- Speech Duration: ~600-700 seconds (70-80
- Number of Segments: 50-100 segments
- Average Segment Length: 6-12 seconds
- Training Samples: 50-100 ASR + Translation pairs

🔮 Future Enhancements

### Planned Features
1. French Translation Integration - Add Bambara ↔ French support
2. Speaker Diarization - Identify different speakers
3. Topic Segmentation - Segment by conversation topics
4. Quality Scoring - Advanced quality metrics
5. Active Learning - Identify segments needing human review
6. Real-time Processing - Stream processing capabilities

### Integration Opportunities
- N'Ko Educational Videos - Apply same pipeline to N'Ko lessons
- Multi-language Support - Extend to other West African languages
- Cloud Processing - Scale to large episode collections
- API Services - Provide processing as a service