Codex Handoff — MotionMix / ShootView / Still-Capture iOS Build

Codex Handoff — MotionMix / ShootView / Still-Capture iOS Build

> Author: Claude (Mac1 session ff0dc14e), 2026-05-03 evening, US ET
> Audience: Codex (next agent, fresh context)
> Operator: Mohamed
> Scope: finish the MotionMix multi-cam shoot stack so ShootView becomes the operator surface, iPhones produce max-resolution stills end-to-end, and Stage View is solid enough to ship.

---

1. Read this first (the architectural decision today)

The user explicitly decided today, 2026-05-03: ShootView (iPad SwiftUI app) absorbs the Live Director role. The native macOS `MotionMixDirector` Swift Package app is no longer the operator surface — it stays buildable as a fallback / debug tool, but ShootView is the canonical operator + photographer + gallery + reel + lookbook UI now.

User quote: "I, in fact, do not need the Live Director app itself, and I could actually direct my shoot through ShootView without any other external things."

Implication for you: do not pour effort into `Desktop/MotionMixDirector/` features. All operator-facing work goes into `Desktop/ShootView/`.

---

2. Machine + service map (current live state)

Use `xcrun devicectl list devices` on Mac1 to resolve full UDIDs when deploying.

---

3. Repos / paths you'll be touching

---

4. What I shipped this session (uncommitted, all on Mac1 main)

4.1 MotionMix iOS still-capture pipeline (yesterday 2026-05-02, 3 files)

The original problem: when the photographer presses the shutter on `shoot.html` (or ShootView), the server fires `POST /capture-photo` and broadcasts `{type: "capture_photo", ...}` over `/device/ws`. iPhones receive it via `DirectorHubClient.onCapturePhotoCommand` callback but the callback was never assigned anywhere, so nothing happened. Stills only existed via `/capture` → `remote_capture` which fetches the live MJPEG `/snapshot` (no Smart HDR / Deep Fusion / Photonic Engine — it's just a video frame).

I wired `AVCapturePhotoOutput` end to end. Three patches:

a. `MotionMixApp/MotionMixApp/Services/CameraService.swift`

• Added `private let photoOutput = AVCapturePhotoOutput()` next to existing `videoOutput`.
• Added `private var photoDelegates: [Int64: PhotoCaptureDelegate]` to retain delegates across capture (delegate gets released after photo callback to free memory).
• Inside `configureSession()`, after `addOutput(videoOutput)`:

  if session.canAddOutput(photoOutput) {
      session.addOutput(photoOutput)
      photoOutput.maxPhotoQualityPrioritization = .quality
      if let activeFormat = session.inputs
          .compactMap({ ($0 as? AVCaptureDeviceInput)?.device.activeFormat })
          .first,
         let maxDims = activeFormat.supportedMaxPhotoDimensions.last {
          photoOutput.maxPhotoDimensions = maxDims
      }
  }

• New `extension CameraService { func captureStill(flash: String) async -> Data? }` — uses HEVC HEIC when available, sets `.quality` prioritization + `maxPhotoDimensions`, supports `flash = "off" | "on" | "auto"`.
• New `final class PhotoCaptureDelegate: NSObject, AVCapturePhotoCaptureDelegate` at file scope — calls `photo.fileDataRepresentation()` in `didFinishProcessingPhoto`.

b. `MotionMixApp/MotionMixApp/Services/DirectorHubClient.swift`

Added near the existing `httpBaseURL` private property:

@discardableResult
func uploadStill(data: Data, score: Float = 0.95, reason: String = "manual") async -> Bool

Builds a multipart body with fields `device_id` (= `self.deviceId`), `device_name` (= `UIDevice.current.name`), `score`, `reason`, and an `image` part with `Content-Type: image/heic`. POSTs to `<httpBaseURL>/still-upload`. Returns `true` on HTTP 200. Logs failures via the existing `writeDebugLog`.

`UIKit` is already imported in this file.

c. `MotionMixApp/MotionMixApp/MotionMixApp.swift`

Wired right above the `// Handle server LiDAR commands` block (search for that comment):

// Handle server capture_photo command (Shoot view shutter / dashboard capture)
let camera = cameraService
directorHub.onCapturePhotoCommand = { [weak directorHub] flash in
    Task { @MainActor in
        guard let data = await camera.captureStill(flash: flash) else {
            print("[CAPTURE] camera.captureStill returned nil")
            return
        }
        let ok = await directorHub?.uploadStill(data: data, score: 0.95, reason: "manual") ?? false
        print("[CAPTURE] uploaded \(data.count) bytes flash=\(flash) ok=\(ok)")
    }
}

Build verification: `xcodebuild build -workspace MotionMixApp.xcworkspace -scheme MotionMixApp -destination 'generic/platform=iOS'` — my three patches compile cleanly; the only error is unrelated (see §5 blocker A).

---

4.2 multicam-server `/shoot` browser page (yesterday 2026-05-02)

`Desktop/MotionMix/multicam-server/src/shoot.html` (new, ~165 lines)

Browser-only fallback for "shoot view" — usable from any iPad/laptop without installing ShootView. Contract:

• Subscribes `/director/ws` → mirrors current cut (whatever camera is active fills the screen as MJPEG).
• Subscribes `/hero-feed/ws` → flashes red `HERO POSE` pill when a hero window opens.
• Big white shutter circle → `POST /capture-photo {"flash":"off"}`.
• Bottom strip: last 12 stills, tap to star/unstar via `POST /still/:id/star/set {starred:bool}`.
• Toast confirmations + haptic feedback (`navigator.vibrate?.(8)`).
• Auto-rotate to landscape via `viewport-fit=cover` + `apple-mobile-web-app-capable`.

`Desktop/MotionMix/multicam-server/src/main.rs` — added the handler near `dashboard()` (~ line 7400):

async fn shoot() -> Html<&'static str> {
    Html(include_str!("shoot.html"))
}

And the route near `/multi-cam` (~line 9921):

.route("/shoot", get(shoot))

`cargo build --release` was run, binary deployed, KeepAlive picked it up. Currently serving at `http://[ip]:9404/shoot` — verified HTTP 200 today.

---

4.3 ShootView force-cut + active-camera tracking (today 2026-05-03)

The original ShootView didn't track the active camera and had no operator-style force-cut. This patch turns ShootView into the operator console.

`ShootView/ShootView/Models/ShootClient.swift`

Added new published state next to `editedCounts`:

@Published var activeCameraId: String?
@Published var directorMode: String?  // "auto" | "manual" | "ghost" | "photoshoot"

In `handleWebSocketMessage`, added a `connected` case (was missing) that mirrors the server's hello payload, and patched `cut` to also update `activeCameraId`:

case "connected":
    if let cam = json["active_camera"] as? String { activeCameraId = cam }
    if let mode = json["mode"] as? String { directorMode = mode }
case "cut":
    if let to = json["to"] as? String {
        activeCameraId = to
        for i in cameras.indices { cameras[i].isOnProgram = (cameras[i].id == to) }
    }

New method (sits right above `// MARK: - Remote Capture`):

func forceCut(to cameraId: String) {
    if isViewingPastSession { return }
    let payload: [String: Any] = ["type": "activate", "camera": cameraId]
    guard let data = try? JSONSerialization.data(withJSONObject: payload),
          let str = String(data: data, encoding: .utf8) else { return }
    webSocketTask?.send(.string(str)) { _ in }
    activeCameraId = cameraId       // optimistic; server `cut` reconciles
    directorMode = "manual"
}

The server's `handle_director_ws` (`multicam-server/src/main.rs:4945`) accepts inbound `{"type": "activate", "camera": "<id>"}` and flips to Manual mode + broadcasts a `cut` event to all subscribers. Verified.

`ShootView/ShootView/Views/RemoteControlView.swift`

`cameraChip` is now tap-to-cut, and renders a "LIVE" badge + pink stroke when `client.activeCameraId == camera.id`:

private func cameraChip(_ camera: CameraInfo) -> some View {
    let isLive = client.activeCameraId == camera.id
    return VStack(spacing: 4) {
        if let snap = cameraSnapshots[camera.id] {
            Image(uiImage: snap)
                .resizable()
                ...
                .overlay(
                    RoundedRectangle(cornerRadius: 4)
                        .strokeBorder(isLive ? Color.pink : Color.white.opacity(0.1),
                                      lineWidth: isLive ? 2 : 1)
                )
                .overlay(alignment: .topLeading) {
                    if isLive {
                        Text("LIVE")
                            ...
                            .background(Color.pink)
                    }
                }
                .onTapGesture { client.forceCut(to: camera.id) }
        }
        ...
    }
}

Not yet built or deployed. ShootView Xcode project hasn't been compiled with these patches. See §6.

---

5. Hard blockers (need fixing before anything else lands)

Blocker A — `LummReceiver.swift` not in MotionMixApp.xcodeproj Sources phase

`xcodebuild` of MotionMixApp fails with:

MotionMixApp.swift:28:45: error: cannot find 'LummReceiver' in scope

The file exists at `Desktop/MotionMixApp/MotionMixApp/Services/LummReceiver.swift` (a `final class LummReceiver: ObservableObject`) but is not in the Sources build phase of `MotionMixApp.xcodeproj` — someone created it on disk without dragging it into Xcode.

Fix (one-time, in Xcode):
1. Open `Desktop/MotionMixApp/MotionMixApp.xcworkspace` in Xcode.
2. Right-click the `Services` group in the Project Navigator → Add Files to "MotionMixApp"…
3. Select `LummReceiver.swift` from disk.
4. Verify Target Membership = MotionMixApp is checked.
5. Cmd-B to confirm clean build.

Until this is done, §4.1 cannot land on the iPhones. Once fixed, re-run my build/deploy commands in §6.

Blocker B — ShootView not deployed to iPad with today's patches

§4.3 patches are on Mac1 disk only. The iPad still runs the previous build, so tapping a camera chip does nothing operator-side and `activeCameraId` is always nil.

Fix: build + install ShootView to iPad (see §6.3).

Blocker C (lower-priority / unrelated to MotionMix)

— Mac4 needs Accessibility permission for `[home-path]` if Mohamed wants Fn-on-Mac4 keyboard dictation. UDP audio bridge is fine; this is just the hotkey side. Defer unless Mohamed asks.

---

6. What you should do, in order

6.1 Fix Blocker A and rebuild MotionMixApp

After Xcode adds `LummReceiver.swift` to the target:

cd Desktop/MotionMixApp
xcodebuild build \
  -workspace MotionMixApp.xcworkspace \
  -scheme MotionMixApp \
  -destination 'generic/platform=iOS' 2>&1 | tail -30

Expect: ` BUILD SUCCEEDED `. If you see other errors, surface them — do not auto-fix unrelated breakage without checking with the user first.

6.2 Deploy MotionMixApp to all three iPhones

Get full UDIDs:

xcrun devicectl list devices 2>&1 | grep -iE "iPhone|iPad"

For each iPhone (16 Plus / 16 Pro Max / 14 Pro Max), and per the device map at `Desktop/MotionMixApp/CLAUDE.md`:

APP=$(ls -dt [home-path] | head -1)
xcrun devicectl device install app --device <UDID> "$APP"
xcrun devicectl device process launch --device <UDID> com.openclaw.MotionMixApp

6.3 Build + deploy ShootView to iPad

ShootView is a separate Xcode project (not workspace). Confirm with `find Desktop/ShootView -name "*.xcodeproj" -maxdepth 2`. Then:

cd Desktop/ShootView
xcodebuild build \
  -project ShootView.xcodeproj \
  -scheme ShootView \
  -destination 'generic/platform=iOS' 2>&1 | tail -30

APP=$(ls -dt [home-path] | head -1)
# Use whichever iPad UDID is the photographer's (1DE6FABC… per CLAUDE.md)
xcrun devicectl device install app --device <iPad-UDID> "$APP"
xcrun devicectl device process launch --device <iPad-UDID> <ShootView-bundle-id>

The bundle id is in `ShootView.xcodeproj/project.pbxproj` (search `PRODUCT_BUNDLE_IDENTIFIER`).

6.4 End-to-end test (in this order)

1. multicam-server alive on Mac1 — `curl -sS http://[ip]:9404/devices | head -c 200`. Expect JSON array (possibly empty until iPhones register).
2. iPhones register — launch MotionMixApp on each iPhone; verify they appear in `/devices` JSON with `enabled: true, stream_ready: true`.
3. Stage iPad — open Safari → `http://[ip]:9404/program` → Add to Home Screen → fullscreen. Should show whatever camera the auto-director cut to.
4. ShootView iPad — launch ShootView app. Expect connection banner clears, camera chips populate in `RemoteControlView`'s strip with live thumbnails. The currently-cut chip has the pink "LIVE" border.
5. Force cut — tap a different camera chip in ShootView. Expect:
- LIVE badge moves immediately (optimistic update).
- Stage iPad's `/program` switches feed within ~1 s.
- Server logs `mode: manual` cut decision.
6. High-quality capture — tap the shutter button in ShootView's `RemoteControlView` (which calls `client.capturePhoto(flash:)` → `POST /capture-photo`). Expect, within ~2-3 s:
- All enabled iPhones capture HEIC at native max res (8064×6048 on Pro Max, 4032×3024 on iPhone 14 Pro Max).
- Each iPhone uploads via multipart `POST /still-upload`.
- Server broadcasts `still_captured` over `/director/ws`.
- ShootView's gallery prepends new stills with animation.
- Browser `/shoot` strip refreshes on its 6-s poll.

If any step fails, surface the failure with logs — do not silently proceed.

6.5 Solidify Stage View (program.html crossfade + connection-lost overlay)

`Desktop/MotionMix/multicam-server/src/program.html` is currently 67 lines. It does hard cuts only and reconnects every 2 s on WS close. Cut events from the server already carry `transition: "crossfade" | "hardCut"` — the page ignores it.

Patch to land:

1. Use two `<img>` layers stacked, alternate which is on top, fade between them when `transition === "crossfade"`. Hard cut = instant swap. Use a 250 ms ease-in-out on `opacity`.
2. Add a connection-lost overlay (semi-opaque dim + small pill at top center: "RECONNECTING…") that shows when the WS hasn't received any message in > 5 s. Hide on next `cut` or `connected`.
3. Keep the existing `fetchDevices` polling and stream-URL builder — don't refactor them.
4. Stay OBS-friendly — black background, no scrollbars, no titlebar.

After the patch:

cd Desktop/MotionMix/multicam-server
cargo build --release 2>&1 | tail -10
launchctl kickstart -k gui/$(id -u)/com.motionmix.multicam-server
curl -sS -o /dev/null -w "program=%{http_code}\n" http://[ip]:9404/program

6.6 Optional: ShootView "release to auto"

Today's `forceCut` flips the server to Manual mode and never goes back. The auto director won't take over. Worth adding a small "↻ Auto" button in `RemoteControlView` that sends `{"type":"set_mode","mode":"auto"}` over the same WS. Server-side check: `handle_director_ws` (`main.rs:4896`) — confirm it accepts `set_mode` (it accepts `activate` already; if not, add a small handler).

---

7. Endpoint reference (so you don't have to grep)

HTTP routes (multicam-server :9404)

WebSockets (`/director/ws`)

Outbound (server → client):

{"type":"connected","active_camera":"mm-iphone-…","mode":"auto","show_cue":{...},"temporal":{...}}
{"type":"cut","from":"...","to":"mm-iphone-…","transition":"crossfade|hardCut","reason":"...","timestamp_ms":...}
{"type":"still_captured","id":"...","device_id":"...","device_name":"...","score":0.95,"reason":"manual","path":"/stills-images/...","thumb_path":"/stills-thumbs/...","timestamp_ms":...}
{"type":"hero_lock","stillness":0.92,...}
{"type":"star_changed","id":"...","starred":true}
{"type":"auto_edit_done","preset":"editorial_grade","count":12}
{"type":"scores","scores":{"mm-iphone-…":0.71,...}}

Inbound (client → server):

{"type":"activate","camera":"mm-iphone-…"}     // operator force-cut, sets mode=manual
// Possibly also: {"type":"set_mode","mode":"auto|manual|ghost|photoshoot"} — verify in handle_director_ws

WebSockets (`/device/ws` — server fans out commands to iOS apps)

{"type":"capture_photo","device":"mm-iphone-…","flash":"off"}
{"type":"record_start","target_device":"mm-iphone-…","camera":"back","timestamp_ms":...}
{"type":"record_stop","target_device":"mm-iphone-…","timestamp_ms":...}
{"type":"set_torch","device":"mm-iphone-…","mode":"on"}
{"type":"flip_camera","device":"mm-iphone-…"}
// + lens / zoom / placement / config events

iOS-side handler: `MotionMixApp/Services/DirectorHubClient.swift` — search for `case "capture_photo"`, `case "record_start"`, etc.

WebSockets (`/hero-feed/ws`)

{"type":"hero_window_open", ...}     // triggers HERO POSE flash in shoot.html

---

8. Build commands (cheat sheet)

---

9. What you must NOT do

• Don't push to remote. All today's work is uncommitted. Mohamed reviews diffs before push. If you commit, stay on `main` locally.
• Don't refactor `multicam-server/src/main.rs`. It's 10k lines and has very specific scoring/timing invariants. Only touch the parts you need (handlers, route table, html files via `include_str!`).
• Don't touch the SAN / Echelon Rust FFI in MotionMixApp (`Services/EchelonBridge.swift`, `SANService.swift`, `Frameworks/libechelon_ios.a`). It's the music-from-motion brain — separate concern from this work.
• Don't switch ShootView's existing `/capture` calls (low-quality MJPEG path) to `/capture-photo` blindly. Keep both: `/capture` is "preview-grade fast", `/capture-photo` is "max quality" via §4.1. The user may want both for different modes.
• Don't attempt the `LummReceiver` pbxproj fix programmatically by editing `project.pbxproj` directly. The XML format is fragile and easy to corrupt. It's a 30-second Xcode GUI action — let Mohamed (or you, prompting Mohamed) do it manually.
• Don't reinstall MotionMixDirector as the operator app. The user explicitly said it's deprecated for ops.
• Don't restart `multicam-server` while iPhones are mid-record — they'll lose the active session. Check `lsof -iTCP:9404 -sTCP:ESTABLISHED` before restarting.

---

10. Open questions for the user (surface as you go)

1. "Release to auto" button placement — top toolbar in ShootView, or footer next to the shutter? (See §6.6.)
2. Should `forceCut` lock the cut for N seconds (hold mode) before auto can take over? Server already supports a `hold_until_ms` field on cuts; could expose a long-press in ShootView for "hold this for 30 s".
3. `score=0.95` on manual captures — is that fine, or should manual captures bypass the scoring/auto-edit pipeline entirely?
4. Stage View resolution / display size — current `program.html` is `object-fit: contain` over the full viewport. If the stage iPad has a specific aspect ratio (e.g., wall projection), we may need to letterbox / crop differently.
5. OBS pickup — `program.html` is OBS-friendly, but is OBS actually wired? `MACOS-LIVE-DIRECTOR-PLAN.md` Wave 3 is "Go Live" hooks; status unknown.

---

11. Memory references

The Claude memory files most relevant to this handoff (all under `[home-path]`):

• `motionmix-shootview-takes-over-live-director-2026-05-03.md` — today's pivot and ShootClient/RemoteControlView patches
• `motionmix-still-capture-build-2026-05-02.md` — yesterday's iOS still-capture wiring (the §4.1 patches)
• `motionmix-stage-shoot-director-2026-05-02.md` — `/shoot` page + multicam-server bring-up
• `MEMORY.md` — index pointing to all of the above

If anything in this handoff conflicts with a memory file, the memory file wins (it was written first-hand with verification).

---

12. Tight definition of done

You're done when:
1. Mohamed taps the shutter button in ShootView on the iPad and 3 iPhones produce HEIC stills at native max resolution within ~2-3 s.
2. The new stills appear in ShootView's gallery and in the `/shoot` browser strip without manual refresh.
3. Tapping a camera chip in ShootView changes what the Stage iPad displays within ~1 s, and the chip shows the LIVE badge.
4. `program.html` survives a multicam-server restart cleanly (reconnects, shows connection-lost overlay during the gap, no white flash on cut transitions).
5. All four points above hold across a 10-minute session without manual intervention.

Ship that, then surface the open questions in §10.

— Claude (handoff Mac1 ff0dc14e, 2026-05-03)