Convex Production Backend Handoff

Convex Production Backend Handoff

Purpose

This is the implementation contract for Claude Code.

The goal is to add a production memory/control plane for the MotionMix AI photoshoot + motion-music lane on top of the existing Convex memory app.

This is backend/API work only.

Do not build frontend UI in this pass.
Do not re-architect MotionMix.
Do not move anything back to Supabase.

The target repo is:

- `Desktop/Comp-Core/apps/convex-memory`

The existing substrate is already live:

• `convex/schema.ts`
• `convex/algorithmMaxing.ts`
• `convex/http.ts`

This work should extend that system, not fork it.

Product Goal

Support a new session type:

• AI-directed multicam photo/video shoot
• pose-aware still extraction
• motion-derived music/session structure

Convex should remember the session so later systems can:

• reconstruct the cut log
• find hero pose windows
• extract still candidates
• align motion and music events
• track final assets

Ground Truth

Existing Algorithm Maxing tables already exist in `schema.ts`:

• `feedTapSessions`
• `feedTapSessionItems`
• `contentItems`
• `contentEmbeddings`
• `reactabilityScores`
• `streamQueue`
• `contentProjectMap`
• `algorithmHealth`

Existing operational mutations/queries already exist in `algorithmMaxing.ts`:

• `upsertContentItem`
• `upsertContentEmbedding`
• `writeReactabilityScore`
• `exportStreamQueuePage`
• `getNextStreamQueueItem`
• `enqueueStreamItem`
• `markStreamQueueItemStatus`
• `writeStreamQueueFeedback`
• `upsertContentProjectMap`

Existing HTTP route style already exists in `http.ts`:

• `/feed-tap/*`
• `/algorithm-maxing/stream/*`

This new production lane should follow those exact patterns.

Scope

Claude Code should implement:

1. Convex schema additions
2. new Convex mutations/queries
3. HTTP actions for those functions
4. type-safe production event contracts
5. verification via deploy + live runs

Claude Code should not implement in this pass:

• still frame extraction workers
• music rendering workers
• OBS integration
• MotionMix frontend changes
• TouchDesigner changes

Those systems will write into this backend later.

File Ownership

Claude Code should edit:

• `Desktop/Comp-Core/apps/convex-memory/convex/schema.ts`
• `Desktop/Comp-Core/apps/convex-memory/convex/http.ts`

Claude Code should add a new module:

- `Desktop/Comp-Core/apps/convex-memory/convex/production.ts`

Do not keep piling everything into `algorithmMaxing.ts`.
That file already owns the content/review lane.
The photoshoot/music session ledger deserves its own module.

Schema To Add

Add the following Convex tables.

1. `productionSessions`

Purpose:

- one row per shoot / live creative session

Fields:

• `sessionType: string`
• expected values initially:
• `photoshoot_performance`
• `photoshoot_review`
• `music_demo`
• `status: string`
• `setup`
• `rolling`
• `paused`
• `complete`
• `cancelled`
• `title: optional string`
• `notes: optional string`
• `directorMode: optional string`
• `motion`
• `pose`
• `reset`
• `manual`
• `musicMode: optional string`
• `live_sketch`
• `capture_only`
• `disabled`
• `projectSlug: optional string`
• `startedAt: number`
• `completedAt: optional number`
• `createdAt: number`
• `updatedAt: number`
• `metadata: optional any`

Indexes:

• `by_status_started`
• `by_session_type_started`
• `by_project_started`

2. `productionCameras`

Purpose:

- inventory of cameras participating in a session

Fields:

• `sessionId: id("productionSessions")`
• `cameraId: string`
• `deviceName: optional string`
• `sourceType: string`
• examples:
• `iphone`
• `insta360_x4`
• `virtual_camera`
• `role: string`
• examples:
• `hero`
• `left_profile`
• `right_profile`
• `detail`
• `wide`
• `context`
• `priorityWeight: float64`
• `isActive: boolean`
• `isWideFallback: optional boolean`
• `positionHint: optional string`
• `createdAt: number`
• `updatedAt: number`
• `metadata: optional any`

Indexes:

• `by_session_camera`
• `by_session_role`
• `by_session_active`

3. `productionDirectorEvents`

Purpose:

- append-only event ledger from multicam-server / director / operator

Fields:

• `sessionId: id("productionSessions")`
• `timestampMs: number`
• `eventType: string`
• initial supported values:
• `cut`
• `hold`
• `release`
• `mode_change`
• `camera_priority_change`
• `hero_pose_marker`
• `still_candidate_marker`
• `music_phrase_start`
• `music_phrase_end`
• `bar_fire`
• `tempo_update`
• `cameraId: optional string`
• `reason: optional string`
• `score: optional float64`
• `metadata: optional any`
• `createdAt: number`

Indexes:

• `by_session_timestamp`
• `by_session_event_type`
• `by_session_camera_timestamp`

4. `productionStillCandidates`

Purpose:

- best frame candidates attached to pose windows

Fields:

• `sessionId: id("productionSessions")`
• `cameraId: string`
• `timestampMs: number`
• `imagePath: optional string`
• `score: float64`
• `poseLabel: optional string`
• `sourceEventId: optional id("productionDirectorEvents")`
• `isFavorite: boolean`
• `selectedForEdit: boolean`
• `createdAt: number`
• `updatedAt: number`
• `metadata: optional any`

Indexes:

• `by_session_score`
• `by_session_timestamp`
• `by_session_camera`

5. `productionMusicMoments`

Purpose:

- store motion-to-music structure during the session

Fields:

• `sessionId: id("productionSessions")`
• `timestampMs: number`
• `section: optional string`
• examples:
• `intro`
• `build`
• `drop`
• `release`
• `tempo: optional float64`
• `confidence: optional float64`
• `phase: optional float64`
• `energy: optional float64`
• `motionSignature: optional string`
• `audioSnapshot: optional any`
• `createdAt: number`
• `metadata: optional any`

Indexes:

• `by_session_timestamp`
• `by_session_section`

6. `productionAssets`

Purpose:

- track exported session outputs

Fields:

• `sessionId: id("productionSessions")`
• `assetType: string`
• examples:
• `video_master`
• `contact_sheet`
• `hero_still`
• `song_sketch`
• `clip`
• `path: string`
• `source: optional string`
• `status: optional string`
• `createdAt: number`
• `updatedAt: number`
• `metadata: optional any`

Indexes:

• `by_session_asset_type`
• `by_session_created`

Backend API Module

Add:

- `convex/production.ts`

This file should own all production-lane mutations and queries.

Mutations to implement

`startProductionSession`

Args:

• `sessionType`
• `title?`
• `notes?`
• `directorMode?`
• `musicMode?`
• `projectSlug?`
• `metadata?`

Behavior:

• insert a new `productionSessions` row
• set `status = "setup"` or `"rolling"` depending on arg or default to `setup`
• return the new id

`updateProductionSession`

Args:

• `sessionId`
• patchable fields above

Behavior:

• patch session
• update `updatedAt`

`completeProductionSession`

Args:

• `sessionId`
• `status?` default `complete`
• `notes?`

Behavior:

• set `completedAt`
• patch status
• update `updatedAt`

`upsertProductionCamera`

Args:

• `sessionId`
• `cameraId`
• `deviceName?`
• `sourceType`
• `role`
• `priorityWeight`
• `isActive`
• `isWideFallback?`
• `positionHint?`
• `metadata?`

Behavior:

• unique by `sessionId + cameraId`
• patch existing or insert new

`setProductionCameraState`

Args:

• `sessionId`
• `cameraId`
• `isActive?`
• `role?`
• `priorityWeight?`
• `metadata?`

Behavior:

- patch existing camera row

`appendDirectorEvent`

Args:

• `sessionId`
• `timestampMs`
• `eventType`
• `cameraId?`
• `reason?`
• `score?`
• `metadata?`

Behavior:

- append one row to `productionDirectorEvents`

`appendDirectorEventsBatch`

Args:

• `sessionId`
• `events: array`

Behavior:

• insert multiple event rows
• keep implementation simple and safe; no need for exotic batching tricks yet

`markHeroPose`

Args:

• `sessionId`
• `timestampMs`
• `cameraId?`
• `reason?`
• `score?`
• `metadata?`

Behavior:

- write a `productionDirectorEvents` row with `eventType = "hero_pose_marker"`

`createStillCandidate`

Args:

• `sessionId`
• `cameraId`
• `timestampMs`
• `imagePath?`
• `score`
• `poseLabel?`
• `sourceEventId?`
• `isFavorite?`
• `selectedForEdit?`
• `metadata?`

Behavior:

- insert still candidate row

`updateStillCandidate`

Args:

• `stillCandidateId`
• `imagePath?`
• `score?`
• `poseLabel?`
• `isFavorite?`
• `selectedForEdit?`
• `metadata?`

Behavior:

- patch row

`appendMusicMoment`

Args:

• `sessionId`
• `timestampMs`
• `section?`
• `tempo?`
• `confidence?`
• `phase?`
• `energy?`
• `motionSignature?`
• `audioSnapshot?`
• `metadata?`

Behavior:

- insert one row in `productionMusicMoments`

`createProductionAsset`

Args:

• `sessionId`
• `assetType`
• `path`
• `source?`
• `status?`
• `metadata?`

Behavior:

- insert asset row

Queries to implement

`getProductionSession`

Args:

- `sessionId`

Behavior:

- return session row

`getActiveProductionSession`

Args:

- optional `sessionType`

Behavior:

- return the latest session where `status` is not complete/cancelled

`listProductionSessionCameras`

Args:

- `sessionId`

Behavior:

- return all session cameras ordered deterministically

`exportProductionEventPage`

Args:

• `sessionId`
• `paginationOpts`
• optional `eventType`
• optional `cameraId`

Behavior:

- paginate `productionDirectorEvents`

`listStillCandidates`

Args:

• `sessionId`
• optional `limit`
• optional `cameraId`
• optional `favoritesOnly`

Behavior:

- return best stills ordered by score desc, then timestamp

`listMusicMoments`

Args:

• `sessionId`
• optional `limit`

Behavior:

- return moments ordered by timestamp

`listProductionAssets`

Args:

• `sessionId`
• optional `assetType`

Behavior:

- return created assets

`getProductionTimeline`

Args:

• `sessionId`
• optional `windowStartMs`
• optional `windowEndMs`

Behavior:

• return:
• session
• cameras
• recent director events
• recent still candidates
• recent music moments

This is the operator/debug query that later tools can consume.

HTTP Actions

Extend `convex/http.ts`.

Add routes under `/production/*`.

Session routes

• `POST /production/session/start`
• `POST /production/session/update`
• `POST /production/session/complete`
• `GET /production/session/active`
• `GET /production/session/timeline`

Camera routes

• `POST /production/camera/upsert`
• `POST /production/camera/state`
• `GET /production/cameras`

Event routes

• `POST /production/event`
• `POST /production/event/batch`
• `GET /production/events`
• `POST /production/hero-pose`

Still routes

• `POST /production/still-candidate`
• `POST /production/still/update`
• `GET /production/stills`

Music routes

• `POST /production/music-moment`
• `GET /production/music-moments`

Asset routes

• `POST /production/asset`
• `GET /production/assets`

Follow the same style as the existing `algorithm-maxing` HTTP actions:

• parse JSON body
• validate required fields
• call `ctx.runMutation` / `ctx.runQuery`
• return `{ ok: true, result }`

Data Contract Notes

Event ledger rules

`productionDirectorEvents` is append-only.
Do not patch historical event rows unless there is a true data correction need.

Time source

Use `Date.now()` at the backend as the default `createdAt` / `updatedAt`.
Allow event timestamps like `timestampMs` to come from the caller when the source system is multicam-server or music timing.

Session status

Use string enums in code comments/docs, but stay consistent in writes.
Do not invent new statuses ad hoc.

Camera identity

Use plain string `cameraId` because multicam-server and virtual-camera sources already live there.
Do not prematurely force those into Convex ids.

Integration Expectations

This backend will later receive writes from:

1. `multicam-server`
- `cut`
- `hold`
- `release`
- mode changes
- camera priority changes

2. MotionMix pose logic
- hero pose markers
- still candidate markers

3. motion-to-music path
- bar fire
- phrase boundaries
- tempo / phase / energy summaries

4. post workers
- still extraction outputs
- song sketch path
- final asset registration

That means Claude Code should design the schema/API for external writers, not only for one internal script.

Verification

Claude Code must verify with the real Convex app.

Required commands:

cd Desktop/Comp-Core/apps/convex-memory
npx tsc --noEmit
npx convex codegen --typecheck disable
npx convex deploy --typecheck disable --env-file .env.local

Then run live smoke tests for:

• start session
• upsert cameras
• append director event
• mark hero pose
• append music moment
• create still candidate
• create production asset
• fetch timeline

Use `npx convex run ...` or HTTP route checks.

Acceptance Criteria

This task is done only when:

1. schema deploy succeeds
2. all new queries/mutations compile
3. HTTP actions compile and deploy
4. a real `productionSessions` row exists
5. at least one camera row exists for that session
6. at least one director event exists
7. at least one still candidate exists
8. at least one music moment exists
9. at least one production asset exists
10. `getProductionTimeline` returns a coherent combined payload

Explicit Non-Goals

Do not:

• redesign Algorithm Maxing tables
• move existing stream queue logic
• build frontend pages in Convex
• implement media extraction workers
• implement Adobe export logic
• implement the full X4 ingest path in this task

This pass is the Convex production ledger only.

One-Line Summary For Claude Code

Build a new `production.ts` module plus schema and HTTP routes so Convex becomes the persistent session/event ledger for the MotionMix AI photoshoot + motion-music pipeline.