Grand Diomande Research · Full HTML Reader

HANDOFF: Mac4 Unity Takeover for Codex

Date: 2026-04-04 Authoring machine: `[home]` on Mac1 Target machine: Mac4 Primary target project: `[home-path]` on Mac4 Companion reference project: `Desktop/DepthReactiveVisuals/` on Mac1

Embodied Trajectory Systems technical note backlog reference score 30 .md

Full Public Reader

HANDOFF: Mac4 Unity Takeover for Codex

Date: 2026-04-04
Authoring machine: `[home]` on Mac1
Target machine: Mac4
Primary target project: `[home-path]` on Mac4
Companion reference project: `Desktop/DepthReactiveVisuals/` on Mac1

1. What This Document Is For

This is the Mac4 takeover brief for Codex.

The goal is not just "continue the project." The goal is:

1. Open the actual Unity project on Mac4.
2. identify every compile/runtime/configuration problem that Claude Code left behind.
3. fix the errors.
4. clean up the questionable or placeholder work.
5. produce a minimal but real Unity scene that runs.

This handoff is intentionally more skeptical than the earlier general `HANDOFF.md`.

That earlier file is useful for narrative context. This file is for execution.

2. Authoring Conditions and Evidence Quality

This handoff was written under the following conditions:

  • The local machine has a partial source bundle at:
  • `[home]/Desktop/DepthReactiveVisuals/`
  • That local bundle contains:
  • `24` C# files
  • `3` compute shaders
  • `2` docs
  • `2` TouchDesigner source files plus the earlier general handoff
  • The local bundle is not a full Unity project:
  • no `Packages/manifest.json`
  • local `ProjectSettings/` exists but is empty
  • no `.unity` scenes
  • no `.vfx` assets
  • no `.prefab` assets
  • no `.asset` assets
  • no `.asmdef` files
  • no `.meta` files
  • Mac4 was not reachable by SSH during this handoff pass.
  • `ssh mac4` timed out.
  • Therefore this document separates:
  • `confirmed now`
  • `claimed previously`
  • `must verify on Mac4 before trusting`

Treat any statement about the live Mac4 Unity project as untrusted until Codex re-verifies it on Mac4.

3. Machine and Project Topology

Mac1 local reference bundle

Use this for source inspection and context:

  • `[home]/Desktop/DepthReactiveVisuals/Assets/`
  • `[home]/Desktop/DepthReactiveVisuals/TouchDesigner/`
  • `[home]/Desktop/DepthReactiveVisuals/Docs/`

This looks like the authored source bundle, not the authoritative Unity project.

Mac4 Unity workspace

Previous session claims say the real Unity project lives at:

- `[home-path]`

Previous session also claimed this Mac4 project has:

  • `Packages/manifest.json`
  • `ProjectSettings/ProjectVersion.txt`
  • all `24` C# scripts
  • all `3` compute shaders
  • Unity 6 / URP / VFX Graph packages

That all needs to be re-verified on Mac4.

Mac5 TouchDesigner companion

Mac5 is now relevant because the TouchDesigner prototype has been brought back to a live state and can serve as a visual reference for Unity.

Relevant Mac5 files:

  • `Desktop/cc-touchdesigner/depth_reactive_builder.py`
  • `Desktop/cc-touchdesigner/depth_fluid_sim.glsl`
  • `Desktop/cc-touchdesigner/cc-echelon.3.toe`

Current state of the Mac5 TD prototype:

  • `/cc/depth_reactive` now builds successfully
  • latest verification images were captured locally during this session:
  • `/tmp/silhouette_mask_verify.jpg`
  • `/tmp/fluid_density_verify.jpg`
  • `/tmp/final_out_verify.jpg`

This matters because Codex on Mac4 should use the TD behavior as a visual reference, not just trust the earlier Unity-side narrative.

4. Confirmed Local Inventory

C# scripts present in the source bundle

Audio

  • `Assets/Scripts/Audio/AudioAnalyzer.cs`
  • `Assets/Scripts/Audio/AudioToFluid.cs`
  • `Assets/Scripts/Audio/AudioToPalette.cs`
  • `Assets/Scripts/Audio/AudioToVFX.cs`

Core

  • `Assets/Scripts/Core/AttractMode.cs`
  • `Assets/Scripts/Core/CalibrationTool.cs`
  • `Assets/Scripts/Core/DepthProcessor.cs`
  • `Assets/Scripts/Core/DepthSource.cs`
  • `Assets/Scripts/Core/DisplayManager.cs`
  • `Assets/Scripts/Core/FluidSimulator.cs`
  • `Assets/Scripts/Core/FrameRecorder.cs`
  • `Assets/Scripts/Core/IdleDetector.cs`
  • `Assets/Scripts/Core/OpticalFlowProcessor.cs`
  • `Assets/Scripts/Core/PoseTriggers.cs`
  • `Assets/Scripts/Core/PresetManager.cs`

Note: earlier handoff said "Core (10 files)" but there are actually `11` files in `Core`.

Network

  • `Assets/Scripts/Network/BridgedAudioReceiver.cs`
  • `Assets/Scripts/Network/BridgedClient.cs`
  • `Assets/Scripts/Network/EchelonReceiver.cs`
  • `Assets/Scripts/Network/MocopiReceiver.cs`
  • `Assets/Scripts/Network/OSCSender.cs`

Visual

  • `Assets/Scripts/Visual/ChoreographyOverlay.cs`
  • `Assets/Scripts/Visual/ColorPaletteManager.cs`
  • `Assets/Scripts/Visual/ParticleBehaviorSwitcher.cs`
  • `Assets/Scripts/Visual/SkeletonParticleEmitter.cs`

Compute shaders present in the source bundle

  • `Assets/Shaders/Compute/DepthReprojection.compute`
  • `Assets/Shaders/Compute/OpticalFlow.compute`
  • `Assets/Shaders/Compute/FluidSim.compute`

TouchDesigner files present in the source bundle

  • `TouchDesigner/depth_fluid_sim.glsl`
  • `TouchDesigner/depth_reactive_builder.py`

Docs present in the source bundle

  • `Docs/CREATIVE-AGENCY-CONCEPT.md`
  • `Docs/CONTENT-STRATEGY-STORYTELLING-REELS.md`
  • `HANDOFF.md`

5. What Is Confirmed Versus Claimed

Confirmed now from files on Mac1

  • The local source bundle contains exactly `24` C# files and `3` compute shaders.
  • There are no local Unity scenes, VFX assets, prefabs, project settings, package files, asmdefs, meta files, or serialized Unity assets in the Mac1 source bundle.
  • `CalibrationTool.cs` no longer calls a nonexistent `SetExtrinsics()` method.
  • `FrameRecorder.cs` uses `UnityEngine.ScreenCapture.CaptureScreenshot(...)`.
  • `DepthSource.cs` exists only as an abstract base class.
  • There are no concrete subclasses of `DepthSource` anywhere in the local source bundle.
  • `ChoreographyOverlay.cs` writes shader globals `_EchelonForceScale` and `_EchelonVorticity`, but no local compute shader or C# script consumes those values.
  • `AudioToPalette.cs` writes `_PaletteSaturationBoost`, but no local consumer was found in the source bundle.

Claimed previously in the earlier handoff, but not re-verified here

  • Mac4 Unity project exists at `[home-path]`
  • Unity version is `6000.0.72f1`
  • `Packages/manifest.json` on Mac4 includes URP 17.0.3, VFX Graph 17.0.3, Input System 1.11.2
  • all `24` C# scripts and `3` compute shaders were already deployed to Mac4
  • `.asmdef` files were removed on Mac4
  • project "should compile clean"

Must verify on Mac4 before trusting

  • whether the project opens at all
  • actual compile errors in the Console
  • package resolution status
  • existence of scenes
  • existence of `.meta` files and GUID stability
  • whether VFX Graph package is installed and resolved
  • whether there are any concrete `DepthSource` implementations on Mac4 that are missing from Mac1
  • whether Claude Code created any scenes or assets directly on Mac4 that never came back to Mac1

6. The Biggest Reality Check for Codex

Do not assume "the Unity project is basically done."

Based on the source bundle, it is not.

What exists is:

  • a script skeleton
  • compute shader skeletons
  • a partially coherent architecture
  • a lot of intended integration seams

What does not exist locally:

  • a full Unity project snapshot
  • any scene
  • any prefab
  • any VFX Graph asset
  • any concrete depth-camera implementation
  • any confirmed installation-ready hierarchy

This is a recovery and hardening job, not a polish pass.

7. Critical Mac4 Questions Codex Must Answer First

Before changing anything significant on Mac4, Codex should answer these:

1. Does `[home-path]` actually exist and open in Unity?
2. What compile errors appear immediately on open?
3. Is `Packages/manifest.json` real, and are URP + VFX Graph installed/resolved?
4. Are there any scenes or assets on Mac4 that do not exist on Mac1?
5. Are there any `.asmdef` files still present?
6. Are there any concrete `DepthSource` implementations present only on Mac4?
7. Are the compute shaders importing cleanly?
8. Are there broken serialized references because the Mac1 source bundle has no `.meta` files?
9. Is there a `ProjectVersion.txt` and does it match the earlier claim?
10. Is the project actually compiling clean, or did Claude Code just stop after fixing the first obvious errors?

8. Known or Highly Likely Cleanup Targets

These are the most important code-quality and architecture gaps visible from the source bundle.

A. No concrete depth input implementation

Evidence:

  • `DepthSource.cs` is abstract.
  • no subclass of `DepthSource` exists in the local source bundle.

Impact:

  • Unity can compile without a concrete depth source, but the whole depth pipeline cannot run.
  • This is the single biggest functional gap.

Likely fix direction:

  • create a `MockDepthSource` first for deterministic editor testing
  • then create a real `Record3DDepthSource` or equivalent once Mac4 package strategy is decided

B. No Unity scenes or serialized assets

Evidence:

- local bundle has no `.unity`, `.prefab`, `.asset`, `.vfx` files.

Impact:

  • there is nothing to run even if scripts compile.
  • all GameObject wiring will have to be created on Mac4 unless hidden scene work exists there already.

Likely fix direction:

  • create one minimal bootstrap scene first
  • do not try to build the whole installation hierarchy blindly

C. Shader globals are currently disconnected from the actual fluid simulator

Evidence:

  • `ChoreographyOverlay.cs` writes:
  • `_EchelonForceScale`
  • `_EchelonVorticity`
  • no local consumer for those values exists

Impact:

- Echelon-driven fluid modulation is effectively a no-op right now.

Likely fix direction:

  • either route those values into `FluidSimulator` fields directly
  • or explicitly add compute shader parameter reads
  • but do not leave the current fake integration in place

D. Palette saturation global appears unused

Evidence:

  • `AudioToPalette.cs` sets `_PaletteSaturationBoost`
  • no consumer found locally

Impact:

- the "bass scrub" feature likely does nothing

Likely fix direction:

  • either implement the feature in a real shader/VFX path
  • or remove/disable the dead code

E. `PresetManager` is mostly a shell

Evidence:

  • it stores hard-coded defaults instead of reading live values from `FluidSimulator` and `IdleDetector`
  • `fluidSimulator` and `idleDetector` references are not meaningfully used
  • loading only applies palette/behavior, not actual fluid values

Impact:

- preset save/load is not truthful

Likely fix direction:

  • either wire real getters/setters for simulator state
  • or scope the system down and document what the presets actually contain

F. `IdleDetector` is still placeholder logic

Evidence:

  • comment says it reads silhouette motion
  • implementation mostly estimates motion from `fluidSimulator.AudioEnergy`
  • `EstimateMotionFromSilhouette()` does not read the silhouette texture at all

Impact:

- attract mode / superhot mode logic is not grounded in actual body motion

Likely fix direction:

  • read optical flow magnitude or a motion summary texture
  • or explicitly downgrade this to a temporary placeholder

G. `AttractMode` and real audio input will fight each other

Evidence:

  • `AttractMode` writes directly into `fluidSimulator.AudioEnergy` and `AudioBass`
  • `AudioToFluid` also writes those values every frame

Impact:

  • whichever component updates later wins
  • behavior will be unstable or misleading

Likely fix direction:

  • separate attract-mode injection from real audio state
  • or centralize parameter blending in one place

H. `MocopiReceiver` and other JSON parsers are brittle

Evidence:

  • manual string splitting
  • no `CultureInfo.InvariantCulture`
  • minimal validation

Impact:

- network parsing may fail on malformed payloads or locale-specific float parsing

Likely fix direction:

  • replace ad-hoc parsing with structured models if time permits
  • or at minimum harden parsing and use invariant culture

I. `SkeletonParticleEmitter` assumes VFX graph properties and methods exist

Evidence:

  • depends on named graphics buffer properties:
  • `SkeletonPositions`
  • `SkeletonVelocities`
  • `JointWeights`
  • depends on a live VFX graph asset that does not exist locally

Impact:

- script may compile but be inert until Codex creates a matching VFX graph

J. The local source bundle has no `.meta` files

Impact:

  • Mac1 cannot be treated as the serialized source of truth for Unity asset references
  • any scene or VFX asset work done directly on Mac4 may have GUID relationships that do not exist locally

Codex should be careful not to destroy working Mac4 asset GUID relationships by re-copying from the bare Mac1 bundle without thought.

9. Previously Claimed Fixes That Codex Should Re-Verify

CalibrationTool fix

Earlier claim:

- removed call to nonexistent `depthProcessor.SetExtrinsics(CameraToWorld)`

Current local evidence:

- true in `CalibrationTool.cs`

What to verify on Mac4:

  • this exact file is deployed there
  • there is no stale older copy

FrameRecorder fix

Earlier claim:

- changed `ScreenCapture.CaptureScreenshot(...)` to `UnityEngine.ScreenCapture.CaptureScreenshot(...)`

Current local evidence:

- true in `FrameRecorder.cs`

What to verify on Mac4:

  • compile is actually clean
  • no package or namespace conflicts remain

`.asmdef` removal

Earlier claim:

- all `.asmdef` files were removed on Mac4 because they caused namespace cross-reference failures

Current local evidence:

- no `.asmdef` files exist in the Mac1 source bundle

What to verify on Mac4:

  • there are truly no lingering asmdefs
  • Unity is not still holding stale generated project files

10. Local Code Notes by System

Depth pipeline

Files:

  • `Assets/Scripts/Core/DepthSource.cs`
  • `Assets/Scripts/Core/DepthProcessor.cs`
  • `Assets/Scripts/Core/OpticalFlowProcessor.cs`
  • `Assets/Shaders/Compute/DepthReprojection.compute`
  • `Assets/Shaders/Compute/OpticalFlow.compute`

Observations:

  • architecture is coherent
  • actual live input implementation is missing
  • likely first runtime path should use `MockDepthSource` before real hardware

Fluid pipeline

Files:

  • `Assets/Scripts/Core/FluidSimulator.cs`
  • `Assets/Shaders/Compute/FluidSim.compute`

Observations:

  • architecture is plausible
  • integration exists for optical flow + silhouette + audio
  • no direct visible hook from Echelon globals
  • needs live scene wiring and likely tuning

Audio pipeline

Files:

  • `Assets/Scripts/Audio/AudioAnalyzer.cs`
  • `Assets/Scripts/Audio/AudioToFluid.cs`
  • `Assets/Scripts/Audio/AudioToPalette.cs`
  • `Assets/Scripts/Audio/AudioToVFX.cs`

Observations:

  • likely compiles
  • some features are dead-end globals
  • needs a real central blending model if attract mode, local audio, and remote audio all coexist

Network pipeline

Files:

  • `Assets/Scripts/Network/BridgedClient.cs`
  • `Assets/Scripts/Network/BridgedAudioReceiver.cs`
  • `Assets/Scripts/Network/MocopiReceiver.cs`
  • `Assets/Scripts/Network/EchelonReceiver.cs`
  • `Assets/Scripts/Network/OSCSender.cs`

Observations:

  • all are present
  • all use ad-hoc parsing
  • these should be treated as "likely compile, not production hardened"

Visual pipeline

Files:

  • `Assets/Scripts/Visual/ColorPaletteManager.cs`
  • `Assets/Scripts/Visual/ParticleBehaviorSwitcher.cs`
  • `Assets/Scripts/Visual/SkeletonParticleEmitter.cs`
  • `Assets/Scripts/Visual/ChoreographyOverlay.cs`

Observations:

  • architecture assumes a VFX Graph layer that does not currently exist locally
  • palette manager is the most self-contained piece
  • choreography overlay contains at least one fake integration seam

11. TouchDesigner Status That Unity Should Know About

This session repaired the Mac5 TouchDesigner depth-reactive builder enough to serve as a working visual reference.

Files updated this session:

  • `[home]/Desktop/DepthReactiveVisuals/TouchDesigner/depth_reactive_builder.py`
  • `[home]/Desktop/DepthReactiveVisuals/TouchDesigner/depth_fluid_sim.glsl`
  • mirrored to:
  • `[home]/Desktop/cc-touchdesigner/`
  • `mac5:Desktop/cc-touchdesigner/`

Current TD state:

  • live build succeeds
  • animated test mode works without LiDAR
  • fluid branch is now visibly nonzero
  • final output is not yet "hero" quality, but it is debugging-grade and alive

Why Codex on Mac4 should care:

  • Unity does not need to guess the intended behavior from scratch anymore
  • if the Unity fluid path is ambiguous, compare against the current TD contour-field behavior first

12. What Codex Should Do First on Mac4

This is the recommended order.

Phase 1: Establish truth

1. Confirm the real project root:
- `[home-path]`
2. Inventory the project:
- `Packages/`
- `ProjectSettings/`
- `Assets/`
- any `.unity`, `.vfx`, `.prefab`, `.asset`, `.asmdef`
3. Open in Unity and record the exact compile errors.
4. Save the Console output somewhere durable.

Phase 2: Make the project compile clean

1. Remove or fix any remaining namespace or package issues.
2. Re-verify the `CalibrationTool.cs` and `FrameRecorder.cs` fixes.
3. Confirm VFX Graph namespace and package resolution.
4. Confirm `UnityEngine.VFX` references compile.
5. Remove or quarantine dead or obviously broken scripts if needed, but do it deliberately.

Phase 3: Make one minimal runnable scene

Target:

  • one scene
  • one camera
  • one bootstrap hierarchy
  • one mock depth source
  • one fluid output path
  • one visible debug result

Do not start with:

  • multiple presets
  • multiple displays
  • full venue logic
  • final VFX polish

Phase 4: Replace fake seams

Highest-value replacements:

1. create `MockDepthSource`
2. make `IdleDetector` read a real motion proxy
3. route Echelon modulation into real simulator parameters
4. either implement or remove dead global shader hooks

Phase 5: Only then add real assets

After compile-clean minimal scene:

  • create VFX Graph placeholder
  • create scene objects and wire references
  • only then test Record3D / real depth path

13. Suggested Minimal Scene Hierarchy for Codex

This is not present locally. It is the recommended first clean reconstruction.

text
MainScene
  AppRoot
    DisplayManager
    PresetManager
    FrameRecorder

  DepthSystem
    MockDepthSource or Record3DDepthSource
    DepthProcessor
    OpticalFlowProcessor
    FluidSimulator
    CalibrationTool
    IdleDetector

  AudioSystem
    AudioSource or AudioListener host
    AudioAnalyzer
    AudioToFluid
    AudioToPalette
    AudioToVFX

  NetworkSystem
    BridgedClient
    BridgedAudioReceiver
    MocopiReceiver
    EchelonReceiver
    OSCSender

  VisualSystem
    ColorPaletteManager
    ParticleBehaviorSwitcher
    SkeletonParticleEmitter
    ChoreographyOverlay
    VisualEffect host
    display quad or output presenter

  Main Camera
  Debug Canvas

Start with `MockDepthSource`. Do not block scene bring-up on Record3D or Orbbec integration.

14. Exact Files Codex Should Inspect Early

Highest priority

  • `Assets/Scripts/Core/DepthSource.cs`
  • `Assets/Scripts/Core/DepthProcessor.cs`
  • `Assets/Scripts/Core/OpticalFlowProcessor.cs`
  • `Assets/Scripts/Core/FluidSimulator.cs`
  • `Assets/Shaders/Compute/DepthReprojection.compute`
  • `Assets/Shaders/Compute/OpticalFlow.compute`
  • `Assets/Shaders/Compute/FluidSim.compute`

Next priority

  • `Assets/Scripts/Visual/ChoreographyOverlay.cs`
  • `Assets/Scripts/Core/IdleDetector.cs`
  • `Assets/Scripts/Core/PresetManager.cs`
  • `Assets/Scripts/Core/AttractMode.cs`
  • `Assets/Scripts/Audio/AudioToPalette.cs`
  • `Assets/Scripts/Visual/SkeletonParticleEmitter.cs`

Mac5 visual reference

  • `Desktop/cc-touchdesigner/depth_reactive_builder.py`
  • `Desktop/cc-touchdesigner/depth_fluid_sim.glsl`

15. Exact Commands Codex Should Run on Mac4

These are suggestions, not mandates.

bash
cd [home-path]

pwd
find . -maxdepth 3 -type d | sort

find Assets -type f | sort
find Assets -type f \( -name '*.unity' -o -name '*.vfx' -o -name '*.prefab' -o -name '*.asset' -o -name '*.asmdef' \) | sort

sed -n '1,200p' Packages/manifest.json
sed -n '1,80p' ProjectSettings/ProjectVersion.txt

rg -n "class .*: DepthSource|_EchelonForceScale|_EchelonVorticity|_PaletteSaturationBoost" Assets

find Assets -name '*.asmdef' -o -name '*.asmdef.meta'

If Unity CLI is available and licensed:

bash
/Applications/Unity/Hub/Editor/6000.0.72f1/Unity.app/Contents/MacOS/Unity \
  -projectPath [home-path] \
  -batchmode \
  -quit \
  -logFile -

If the exact editor path differs, adjust accordingly.

16. Things Codex Should Not Assume

  • do not assume the Mac4 project matches the Mac1 source bundle
  • do not assume compile-clean status from the earlier handoff
  • do not assume any VFX Graph asset exists
  • do not assume any real depth input implementation exists
  • do not assume preset saving/loading is functional
  • do not assume Echelon modulation is real just because the architecture says so

17. Recommended Acceptance Criteria for the Mac4 Pass

The Mac4 Unity pass should not be considered complete until all of these are true:

1. project opens on Mac4 without package-resolution chaos
2. compile errors are zero
3. there is at least one real `.unity` scene
4. there is at least one runnable path using a mock depth source
5. compute shaders dispatch without obvious null-reference setup failures
6. there is at least one visible render result in Unity
7. dead integration seams are either implemented or explicitly removed
8. Codex leaves behind a cleaner project than the one it inherited

18. The Most Likely Successful Strategy

If Codex tries to "finish the whole vision" in one pass, it will probably waste time.

The better strategy is:

1. recover the true Mac4 project state
2. make compile clean
3. create a mockable minimal scene
4. replace fake seams with real ones
5. only then do polish and feature extension

19. References

Earlier general context:

- `[home]/Desktop/DepthReactiveVisuals/HANDOFF.md`

Agency concept:

- `[home]/Desktop/DepthReactiveVisuals/Docs/CREATIVE-AGENCY-CONCEPT.md`

Content strategy:

- `[home]/Desktop/DepthReactiveVisuals/Docs/CONTENT-STRATEGY-STORYTELLING-REELS.md`

TouchDesigner source:

  • `[home]/Desktop/DepthReactiveVisuals/TouchDesigner/depth_reactive_builder.py`
  • `[home]/Desktop/DepthReactiveVisuals/TouchDesigner/depth_fluid_sim.glsl`

Latest TD verification images from this session:

  • `/tmp/silhouette_mask_verify.jpg`
  • `/tmp/fluid_density_verify.jpg`
  • `/tmp/final_out_verify.jpg`

20. Bottom Line

The useful truth is:

  • there is enough structure here to recover the Unity project
  • there is not enough verified state to trust that Claude Code already "finished" the hard part

Codex on Mac4 should treat this as a recovery, cleanup, and scene-bootstrap operation grounded in real compile/runtime verification.

Promotion Decision

Keep in the searchable backlog until it intersects a live paper or system.

Source Anchor

DepthReactiveVisuals/HANDOFF-MAC4-CODEX-UNITY-TAKEOVER.md

Detected Structure

Method · References · Code Anchors · Architecture