Bambara ASR Integration

Bambara ASR Integration

This integration uses the RobotsMali NVIDIA NeMo models for Bambara automatic speech recognition, complementing our English↔Bambara translation system.

🎯 What This Provides

Complete Speech-to-Speech Pipeline

Bambara Speech → ASR → Bambara Text → Translation → English Text → TTS → English Speech
English Speech → ASR → English Text → Translation → Bambara Text → TTS → Bambara Speech

### Available Models
| Model | Architecture | Parameters | WER | Description |
|-------|-------------|------------|-----|-------------|
| QuartzNet | QuartzNet-15x5 | 19M | 46.5
| Soloni | FastConformer-TDT-CTC | 114M | 40.6

🚀 Quick Start

1. Install Dependencies

# Automatic installation
python install_asr_dependencies.py

# Manual installation
pip install nemo_toolkit[asr] torchaudio librosa soundfile

2. Test the Setup

python test_bambara_asr.py

### 3. Add Audio Files
Place Bambara audio files in the `audio_samples/` directory:
- Formats: WAV, MP3, FLAC, OGG, M4A
- Language: Bambara speech
- Quality: Clear speech, minimal noise

4. Transcribe Audio

# Basic transcription
python use_bambara_asr.py audio_samples/bambara_audio.wav

# With preprocessing and info
python use_bambara_asr.py audio_samples/bambara_audio.wav --preprocess --info

# Use different model
python use_bambara_asr.py audio_samples/bambara_audio.wav --model soloni

# Benchmark performance
python use_bambara_asr.py audio_samples/bambara_audio.wav --benchmark 5

📁 File Structure

src/asr_integration/
├── __init__.py              # Module initialization
├── bambara_asr.py          # Main ASR implementation
└── audio_utils.py          # Audio processing utilities

audio_samples/              # Directory for audio files
├── README.md              # Audio requirements and examples
└── [your_audio_files]     # Place Bambara audio here

# Usage scripts
test_bambara_asr.py        # Test ASR setup
use_bambara_asr.py         # Transcribe audio files
install_asr_dependencies.py # Install dependencies

🔧 Technical Details

### Models Used
- Base Models: RobotsMali's fine-tuned NVIDIA NeMo models
- Training Data: 37 hours of Bambara speech (bam-asr-all dataset)
- Framework: NVIDIA NeMo toolkit
- License: CC-BY-4.0

### Audio Processing
- Input: 16kHz mono WAV files (auto-converted if different)
- Preprocessing: Resampling, normalization, format conversion
- Validation: Audio quality checks and recommendations

### Performance
- QuartzNet: 46.5
- Soloni: 40.6
- GPU Acceleration: Supported for faster processing

💡 Integration with Translation

Complete Pipeline Example

# 1. ASR: Bambara speech → Bambara text
from asr_integration.bambara_asr import BambaraASR
asr = BambaraASR(model_name="soloni")
asr_result = asr.transcribe_file("bambara_speech.wav")
bambara_text = asr_result.text

# 2. Translation: Bambara text → English text
from models.en_bam_translator import EnglishBambaraTranslator
translator = EnglishBambaraTranslator()
translation = translator.translate(bambara_text, "bam", "en")
english_text = translation["translation"]

# 3. TTS: English text → English speech (future integration)
# Complete speech-to-speech translation!

### Use Cases
- Language Learning: Practice Bambara pronunciation
- Communication: Bridge Bambara and English speakers
- Documentation: Transcribe Bambara interviews/recordings
- Research: Analyze Bambara speech patterns

📊 Expected Results

### Sample Transcriptions
| Audio Input | Expected Transcription |
|-------------|----------------------|
| "I ni ce" | i ni ce |
| "I ni sogoma" | i ni sogoma |
| "I ka kene wa?" | i ka kene wa |
| "N tɔgɔ ye Mohamed ye" | n togo ye mohamed ye |

### Performance Metrics
- Accuracy: 40-47
- Speed: 1-3x real-time depending on model and hardware
- Languages: Bambara only (trained specifically for Bambara)

🎤 Recording Tips

For best transcription results:

### Audio Quality
- Use a good microphone (headset or USB mic)
- Record in a quiet environment
- Speak clearly at normal pace
- Avoid background noise, music, or echo

### Content
- Use natural Bambara speech
- Include common phrases and greetings
- Vary speakers (male/female, different ages)
- Mix formal and informal language

### Technical
- Record at 16kHz or higher sample rate
- Use mono (single channel) recording
- Keep files under 5 minutes for processing
- Save as WAV format when possible

🔗 References

• Models: [RobotsMali on Hugging Face](https://huggingface.co/RobotsMali)
• Technical Report: [Weights & Biases Report](https://wandb.ai/yacoudiarra-wl/bam-asr-nemo-training/reports/Draft-Technical-Report-V1--VmlldzoxMTIyOTMzOA)
• Source Code: [Bambara ASR GitHub](https://github.com/diarray-hub/bambara-asr)
• Dataset: [bam-asr-all on Hugging Face](https://huggingface.co/datasets/RobotsMali/bam-asr-all)
• NVIDIA NeMo: [NeMo Toolkit](https://github.com/NVIDIA/NeMo)

🆘 Troubleshooting

Installation Issues

# If NeMo installation fails
pip install --upgrade pip setuptools wheel
pip install nemo_toolkit[asr] --no-cache-dir

# For Apple Silicon Macs
conda install pytorch torchaudio -c pytorch
pip install nemo_toolkit[asr]

### Model Loading Issues
- Check internet connection (models download from Hugging Face)
- Try CPU-only mode: `--device cpu`
- Clear cache: `rm -rf [home-path]`

### Audio Issues
- Use `--info` flag to check audio properties
- Use `--preprocess` flag to optimize audio format
- Ensure audio contains Bambara speech (not other languages)

### Performance Issues
- Use QuartzNet for faster processing
- Use GPU if available: `--device cuda`
- Process shorter audio clips (< 1 minute)

🎉 Success!

You now have a complete Bambara ASR system that:
- ✅ Uses state-of-the-art NVIDIA NeMo models
- ✅ Supports multiple model architectures
- ✅ Handles various audio formats automatically
- ✅ Provides detailed performance metrics
- ✅ Integrates seamlessly with translation
- ✅ Includes comprehensive testing and validation

This creates the foundation for a complete multilingual communication system! 🌍