Mobile Client Specification - Episode 1

Mobile Client Specification - Episode 1

Version: 1.0
Date: November 2025
Status: Implementation Ready

Overview

This document specifies requirements for mobile sensor clients (iOS/Android) that stream data to the Episode 1 host system via WebSocket.

EP1.md: §2, §7 - Mobile client requirements

---

1. Sensor Configuration

1.1 Required Sensors

1.2 Coordinate Systems

Accelerometer/Gyro:
- Use device coordinate system
- X: Right
- Y: Up
- Z: Forward (out of screen)

Gravity:
- If available from device, send as-is
- Otherwise, host will estimate via low-pass filter

---

2. WebSocket Protocol

2.1 Connection

ws://[HOST_IP]:[PORT]

Default port: `8765`

Connection Flow:
1. Client connects to WebSocket
2. Send first packet with `device_id`
3. Host acknowledges and begins logging
4. Stream packets continuously

2.2 Message Format

JSON Structure:

{
  "device_id": "left_phone",
  "ts_device_ms": 1699200000123,
  "ts_wall_ms": 1699200005123,
  "sensors": {
    "accel": {"x": 0.0123, "y": -0.981, "z": 0.034},
    "gyro": {"x": -0.003, "y": 0.014, "z": 0.001},
    "gravity": {"x": 0.0, "y": -9.81, "z": 0.0},
    "baro": 1005.3,
    "mic_rms": 0.023,
    "gps": {
      "lat": 40.7489,
      "lon": -73.9864,
      "speed_mps": 1.2,
      "bearing_deg": 87.3,
      "alt_m": 9.4
    }
  }
}

Field Descriptions:

• `device_id` (string, required): Unique device identifier or role (`"left"`, `"right"`)
• `ts_device_ms` (number, required): Device monotonic timestamp (ms since boot or epoch)
• `ts_wall_ms` (number, optional): Wall-clock timestamp (Unix epoch ms)
• `sensors` (object, required): Sensor data
• `accel` (object, required if available): {x, y, z} in m/s²
• `gyro` (object, required if available): {x, y, z} in rad/s
• `gravity` (object, optional): {x, y, z} in m/s²
• `baro` (number, optional): Pressure in hPa
• `mic_rms` (number, optional): Microphone RMS (0-1 normalized or Pa)
• `gps` (object, optional): Location data

Null/Missing Fields:
- If a sensor is unavailable, omit the field or set to `null`
- Host will handle missing data gracefully

---

3. Timing & Synchronization

3.1 Timestamp Requirements

Device Timestamp (`ts_device_ms`):
- Must be monotonic (never decreases)
- Prefer device monotonic clock (e.g., `SystemClock.elapsedRealtimeNanos()` on Android)
- If monotonic clock unavailable, use Unix epoch ms
- Convert nanoseconds to milliseconds: `ns / 1_000_000`

Wall Clock Timestamp (`ts_wall_ms`):
- Unix epoch milliseconds
- Used for cross-device sync if available
- Optional but recommended

3.2 Clock Drift

Host implements drift compensation using linear regression:
- Warm-up period: ~2-5 seconds (200-500 packets)
- Expected drift tolerance: ±2 ms/s
- If drift exceeds threshold, host will warn

Client should:
- Use stable clock source
- Avoid clock jumps (e.g., manual time changes)
- If clock jumps detected, send a warning in `sensors` (custom field)

---

4. Packet Frequency & Buffering

4.1 Target Rates

4.2 Packet Size

• Target: <500 bytes/packet
• Max: 1 KB
• Compression: Not required (WebSocket handles efficiently)

4.3 Buffering & Backpressure

Client should:
- Send packets immediately (no buffering)
- If WebSocket send buffer fills, drop oldest packets
- Log packet drops and send count in next successful packet (custom field)

---

5. Connection Recovery

5.1 Reconnection

If connection lost:
1. Wait 1 second
2. Attempt reconnect
3. Exponential backoff: 1s, 2s, 4s, 8s, max 30s
4. Reset device timestamp offset on reconnect

5.2 Error Handling

Client should handle:
- Network interruptions
- Host unavailable
- Invalid JSON responses (log and continue)

---

6. Battery & Performance

6.1 Power Management

• Use batch sensor APIs where available (iOS: `CMMotionManager`, Android: `SensorManager` batch mode)
• Target battery drain: <10
• Allow background execution for continuous capture

6.2 CPU/Memory

• Target CPU: <5
• Target memory: <50 MB
• Avoid allocation spikes (pre-allocate buffers)

---

7. Security & Privacy

7.1 Data Privacy

• Microphone: Only send RMS envelope, not raw audio
• GPS: User must consent before sending location
• Store no data on device (stream-only)

7.2 Network Security

• Use `ws://` for local network (Episode 1)
• For production, upgrade to `wss://` (WebSocket Secure)
• Validate server certificate if using `wss://`

---

8. Testing & Validation

8.1 Test Scenarios

Client must pass:
1. Steady stream: 100 Hz for 60 seconds, <1
2. Reconnect: Disconnect and reconnect within 5 seconds
3. Sensor dropout: Handle unavailable sensors gracefully
4. Low battery: Maintain rate at <20
5. Background mode: Continue streaming when app backgrounded (iOS/Android)

8.2 Diagnostic Mode

Client should provide:
- Live packet rate display
- Drift estimate (if available from host)
- Packet drop count
- Battery consumption

---

9. Reference Implementations

9.1 iOS (Swift)

import CoreMotion
import Starscream // WebSocket library

class SensorClient: WebSocketDelegate {
    let motionManager = CMMotionManager()
    var socket: WebSocket!

    func start(host: String, port: Int, deviceId: String) {
        // Configure motion manager
        motionManager.accelerometerUpdateInterval = 0.01 // 100 Hz
        motionManager.gyroUpdateInterval = 0.01

        // Connect WebSocket
        var request = URLRequest(url: URL(string: "ws://\(host):\(port)")!)
        socket = WebSocket(request: request)
        socket.delegate = self
        socket.connect()

        // Start motion updates
        motionManager.startDeviceMotionUpdates(to: .main) { [weak self] motion, error in
            guard let motion = motion else { return }
            self?.sendPacket(motion: motion, deviceId: deviceId)
        }
    }

    func sendPacket(motion: CMDeviceMotion, deviceId: String) {
        let packet: [String: Any] = [
            "device_id": deviceId,
            "ts_device_ms": Date().timeIntervalSince1970 * 1000,
            "sensors": [
                "accel": [
                    "x": motion.userAcceleration.x,
                    "y": motion.userAcceleration.y,
                    "z": motion.userAcceleration.z
                ],
                "gyro": [
                    "x": motion.rotationRate.x,
                    "y": motion.rotationRate.y,
                    "z": motion.rotationRate.z
                ],
                "gravity": [
                    "x": motion.gravity.x,
                    "y": motion.gravity.y,
                    "z": motion.gravity.z
                ]
            ]
        ]

        if let jsonData = try? JSONSerialization.data(withJSONObject: packet),
           let jsonString = String(data: jsonData, encoding: .utf8) {
            socket.write(string: jsonString)
        }
    }
}

9.2 Android (Kotlin)

import android.hardware.Sensor
import android.hardware.SensorEvent
import android.hardware.SensorEventListener
import android.hardware.SensorManager
import okhttp3.WebSocket
import okhttp3.WebSocketListener

class SensorClient(
    private val sensorManager: SensorManager,
    private val deviceId: String
) : SensorEventListener {

    private var webSocket: WebSocket? = null
    private val startTime = SystemClock.elapsedRealtimeNanos()

    fun start(host: String, port: Int) {
        // Connect WebSocket
        val client = OkHttpClient()
        val request = Request.Builder()
            .url("ws://$host:$port")
            .build()

        webSocket = client.newWebSocket(request, object : WebSocketListener() {
            override fun onOpen(webSocket: WebSocket, response: Response) {
                registerSensors()
            }
        })
    }

    private fun registerSensors() {
        sensorManager.getDefaultSensor(Sensor.TYPE_ACCELEROMETER)?.let {
            sensorManager.registerListener(this, it, 10_000) // 100 Hz
        }
        sensorManager.getDefaultSensor(Sensor.TYPE_GYROSCOPE)?.let {
            sensorManager.registerListener(this, it, 10_000)
        }
    }

    override fun onSensorChanged(event: SensorEvent) {
        val tsDeviceMs = (SystemClock.elapsedRealtimeNanos() - startTime) / 1_000_000

        val packet = JSONObject().apply {
            put("device_id", deviceId)
            put("ts_device_ms", tsDeviceMs)
            put("sensors", JSONObject().apply {
                when (event.sensor.type) {
                    Sensor.TYPE_ACCELEROMETER -> put("accel", JSONObject().apply {
                        put("x", event.values[0])
                        put("y", event.values[1])
                        put("z", event.values[2])
                    })
                    Sensor.TYPE_GYROSCOPE -> put("gyro", JSONObject().apply {
                        put("x", event.values[0])
                        put("y", event.values[1])
                        put("z", event.values[2])
                    })
                }
            })
        }

        webSocket?.send(packet.toString())
    }
}

---

10. Deployment Checklist

Before Episode 1 launch:

• [ ] Client can stream at 100 Hz for 30 minutes
• [ ] Packet loss <1
• [ ] Reconnect works within 5 seconds
• [ ] Battery drain <10
• [ ] Timestamps monotonic (no jumps)
• [ ] GPS permission implemented
• [ ] Background mode works
• [ ] Diagnostic UI shows packet rate

---

11. Future Enhancements (Episode 2+)

• Voice command integration
• On-device ML inference
• Adaptive sample rate based on motion intensity
• Peer-to-peer sync (device-to-device without host)

---

Support

For issues or questions:
- GitHub: [computational-choreography/issues](https://github.com/Diomandeee/computational-choreography)
- Email: [email]