E612 Fluid Sim Unity Implementation Guide

E612 Fluid Sim Unity Implementation Guide

Goal: make the LUME Unity build visually match the direction of the Duncan Fewkes `E612: "Fluid Sim Presets Test"` reference while staying original to LUME.

The target is not a frame-by-frame copy. The target is the system language:

• dark realtime room
• readable body silhouette
• dense but controlled particle/fluid motion
• visible preset switching
• small debug/test-bench HUD for build-in-public clips
• clean venue mode with the HUD hidden
• 1920x440 bar-display-safe composition

Approval mockup:

`[home]/Desktop/lume-commerce/viz/mockups/e612-fluid-sim-lume-mockup.html`

Static screenshot:

`[home]/Desktop/lume-commerce/viz/mockups/e612-fluid-sim-lume-mockup.png`

Related planning note:

`[home]/Desktop/lume-commerce/docs/research/duncan-v3/E612-fluid-sim-mimic-plan.md`

Current Unity Status

This is already mostly built. Do not start from a blank Unity scene.

Existing project:

`[home]/Desktop/lume-commerce/viz/lume-pcloud`

Required Unity host:

Mac4, Unity `6000.3.2f1`, URP/VFX Graph `17.0.4`.

Existing runtime pieces:

Existing FluidSim presets:

• `Default`
• `Default_Smooth`
• `LongThrow`
• `MidThrow`
• `ShortThrow`
• `InvertedObstacle`
• `FrozenVelocityField`
• `None`

Recommended first LUME defaults:

Implementation Strategy

There are two implementation levels.

Level 1: use the existing code-driven overlay.

This is the practical route for now. It uses `LumeFlowParticleOverlay` and can be tuned immediately in the Inspector and F1 picker. This should be the first version we record and approve.

Level 2: author the real VFX Graph.

This is the richer long-term route. It uses `Assets/VFX/LumeFlowParticles.vfx` and `LumeVfxRuntimeBridge`. The bridge is already written, but the graph asset itself must be created in the Unity GUI.

Do Level 1 first. Only move to Level 2 after the look is approved.

Phase 0: Open And Verify The Unity Project

On Mac4:

1. Open Unity Hub.
2. Open project:

`[home]/Desktop/lume-commerce/viz/lume-pcloud`

3. Confirm the editor version is `6000.3.2f1`.
4. Wait for package import.
5. Open the main scene:

`Assets/Scenes/LumeMain.unity`

6. Run:

`Lume > Verify Wave 1+2+3 wiring`

Expected console result:

• `LumeUdpReceiver` present
• `LumePointRenderer` present
• `LumeDepthReprojector` present
• `LumeOpticalFlow` present
• `LumeFluidSim` present
• `LumeAudioFftReceiver` present
• `LumeTransientForcePusher` present
• `LumeFlowParticleOverlay` present
• `LumeVfxEditor` present
• `LumeProductionHud` present

If anything is missing, run:

`Lume > Auto-Wire Wave 1+2+3 (selected or LumeMain)`

Then run verify again.

Phase 1: Refresh Presets

Run:

`Lume > Generate Duncan Presets`

This populates:

• `Assets/Resources/LumePresets/Vfx`
• `Assets/Resources/LumePresets/FluidSim`
• `Assets/Resources/LumePresets/Lighting`
• `Assets/Resources/LumePresets/Environment`
• `Assets/Resources/LumePresets/Avatar`
• `Assets/Resources/LumePresets/MANIFEST.md`

Verify these files exist:

• `Assets/Resources/LumePresets/FluidSim/default.asset`
• `Assets/Resources/LumePresets/FluidSim/default-smooth.asset`
• `Assets/Resources/LumePresets/FluidSim/long-throw.asset`
• `Assets/Resources/LumePresets/FluidSim/mid-throw.asset`
• `Assets/Resources/LumePresets/FluidSim/short-throw.asset`
• `Assets/Resources/LumePresets/FluidSim/inverted-obstacle.asset`

Phase 2: Produce Synthetic Input

Before using real hardware, use synthetic publishers so the Unity scene can be tuned without Femto/Mocopi setup friction.

From `[home]/Desktop/lume-commerce`, run these in separate terminals:

python3 services/audio-pub/audio_pub.py --synthetic --bpm 120
python3 services/mocopi-synth/mocopi_synth.py --motion dance --sync-to-lumf
python3 services/depth-pub/depth_pub.py --synthetic-depth

Optional packet check:

python3 packages/lume-tools/packet_inspector.py

Expected traffic:

Phase 3: Tune The Existing Overlay

Select `LumeMain` in the scene.

Confirm these components are on the same root object:

• `LumePointRenderer`
• `LumeOpticalFlow`
• `LumeFluidSim`
• `LumeTransientForcePusher`
• `LumeFlowParticleOverlay`
• `LumeVfxEditor`
• `LumeProductionHud`

Press Play.

Use the F1 picker:

• VFX slot: start with `E606 Maximalism`, `E512 Swirly Particles`, or `E538 Motion Sparks`
• FluidSim slot: test `Default`, `Default_Smooth`, `LongThrow`, `MidThrow`, `ShortThrow`, `InvertedObstacle`
• Lighting slot: start with `FogSpotLeftRight1` or `EmissiveOnly`
• Environment slot: start with `Test Room Dark`

The E612-style preset test should be recorded as a sequence, not a single mode.

Recommended sequence:

1. `Default`
2. `Default_Smooth`
3. `LongThrow`
4. `MidThrow`
5. `ShortThrow`
6. `InvertedObstacle`

Record 8-10 seconds per preset.

Overlay Starting Values

Use these as starting points in `LumeFlowParticleOverlay` if the default look is too far from the mockup.

Do not start with `200000` particles on K11. The reference can run on stronger hardware, but LUME has to stay product-stable.

Phase 4: FluidSim Tuning

`LumeFluidSim` publishes:

• `_FluidDensity`
• `_FluidVelocity`
• `_FluidActive`

It reads:

• `_FlowField`
• `_AudioLevels`
• `_LumeTimeScale`

Starting runtime values:

Preset behavior targets:

Important current limitation:

`InvertedObstacle` currently applies preset dynamics and logs that full depth-mask inversion is a later pass. Do not sell it as fully implemented until the mask inversion path is actually built.

Phase 5: Make The Body Readable

This is the most important visual rule.

The body must remain visible even when the fluid/particles are dense.

Use these controls first:

Acceptance rule:

At any time in the clip, a viewer should be able to point to the body silhouette in under one second.

Phase 6: Bar Display Crop

The mockup is 16:9, but LUME has a 1920x440 bar display.

Do not assume the 16:9 composition works on the bar.

For bar-display mode:

• keep the primary body/action centered horizontally
• keep the silhouette upper body within the middle 60 percent of the height
• keep debug HUD hidden
• keep the fluid response readable at 440px height
• avoid small captions or tiny labels
• use the bar as an inscription/status/reactive strip, not a full UI surface

For build-in-public full-room clips:

• show the debug HUD
• show the F1 picker only briefly
• record the preset label transition
• keep the stage, body, and particles visible

Phase 7: Optional VFX Graph Version

Only do this after the overlay version is approved.

Manual asset:

`Assets/VFX/LumeFlowParticles.vfx`

Recipe:

`Assets/VFX/README-LumeFlowParticles.md`

Required exposed parameters:

After creating the graph:

1. Run `Lume > Bootstrap Scene > Add VFX Graph Particles`.
2. Run `Lume > Verify VFX wiring`.
3. Console should show every exposed parameter as bound.

If `Positions` does not bind, verify the graph parameter type is `GraphicsBuffer`, not `ComputeBuffer`.

Phase 8: Recording Protocol

Record three clips.

Clip A: Preset Test Bench

Purpose: mimic the E612 test structure.

Settings:

• Debug HUD visible
• F1 picker briefly visible
• synthetic depth/audio/mocopi
• cycle through `Default`, `Default_Smooth`, `LongThrow`, `MidThrow`, `ShortThrow`, `InvertedObstacle`

Deliverable:

`E612_LUME_preset_testbench_YYYY-MM-DD.mov`

Clip B: Venue Mode

Purpose: what a venue buyer sees.

Settings:

• HUD hidden
• F1 picker hidden
• `MidThrow`
• `FogSpotLeftRight1` or `EmissiveOnly`
• bar-display preview checked

Deliverable:

`LUME_venue_midthrow_YYYY-MM-DD.mov`

Clip C: Cultural Reveal

Purpose: N'Ko / wordmark direction.

Settings:

• `InvertedObstacle`
• `E544 Logo Testing` or nearest wordmark preset
• warm yellow/gold accents
• lower particle count

Deliverable:

`LUME_nko_inverted_obstacle_YYYY-MM-DD.mov`

Phase 9: Performance Gates

Target hardware:

K11, Ryzen 9 8945HS, Radeon 780M, 32GB.

Minimum acceptance:

If FPS drops:

1. lower `maxParticles`
2. lower `echoPasses`
3. lower `ribbonStretch`
4. set `simulationFrameInterval` to `3`
5. set `gridSize` to `96` or `64`
6. switch from `LongThrow` to `MidThrow`

Do not reduce silhouette readability to chase density.

Phase 10: Real Hardware Bringup

After synthetic clips pass:

1. Connect Orbbec Femto Mega.
2. Start depth publisher:

   python3 services/depth-pub/depth_pub.py --raw-depth

3. Connect UMA-8 or active audio input.
4. Start audio publisher:

   python3 services/audio-pub/audio_pub.py --device "UMA-8"

5. Start Sony/Mocopi path when ready:

   python3 services/sony-bridge/sony_to_lumm_bridge.py

6. Open Unity Play Mode.
7. Toggle F4 production HUD.
8. Confirm:
- depth packets live
- audio packets live
- mocopi packets live or synthetic fallback active
- FluidSim active
- F1 preset changes apply live

Implementation Backlog

These are not required for the first approved E612-style clip, but they are the next real improvements.

Backlog 1: Full InvertedObstacle Mask

Current state:

`LumeFluidSim` logs that full depth-mask inversion is reserved for the next mask pass.

Needed:

• feed a depth/body mask into `LumeFluidSim.compute`
• add branch for `LumeFluidMaskMode.InvertedObstacle`
• inject velocity outside the body instead of through it
• preserve a readable hole-punch body silhouette

Acceptance:

• `InvertedObstacle` visibly routes fluid around the body
• body negative space stays clean
• no new GPU errors on K11

Backlog 2: E612 Mode Preset

Create a `LumeModePreset` named:

`E612 Fluid Sim Preset Test`

Suggested slots:

Acceptance:

• F1 picker can select the full mode with one click
• mode applies VFX, FluidSim, lighting, environment together

Backlog 3: VFX Graph Asset

Build `LumeFlowParticles.vfx` from the existing README.

Acceptance:

• `Lume > Verify VFX wiring` binds all parameters
• particle response is at least as readable as the overlay
• no material/graph import instability

Backlog 4: Bar Display Preview Toggle

Add a quick 1920x440 composition preview mode to F12 or F4 if not already sufficient.

Acceptance:

• one key shows the bar crop
• no small text in crop
• action remains centered

Final Approval Checklist

Before calling this visually approved:

• [ ] Unity opens on Mac4 without package errors.
• [ ] `Lume > Verify Wave 1+2+3 wiring` passes.
• [ ] `Lume > Generate Duncan Presets` produces/updates assets.
• [ ] Synthetic depth/audio/mocopi streams are live.
• [ ] F1 picker opens and applies VFX + FluidSim presets.
• [ ] `Default`, `Default_Smooth`, `LongThrow`, `MidThrow`, `ShortThrow`, and `InvertedObstacle` are visibly different.
• [ ] Body silhouette remains readable in all six presets.
• [ ] Bar-display crop is readable at 1920x440.
• [ ] `MidThrow` is recorded as the proposed LUME default.
• [ ] `ShortThrow` is recorded as the multi-person safety default.
• [ ] `InvertedObstacle` is recorded as the N'Ko/wordmark direction, with current limitation noted.
• [ ] K11 performance does not fall below the acceptance gate.

One-Sentence Implementation Summary

Use the existing `LumeFlowParticleOverlay` plus `LumeFluidSim` preset family first, record the E612-style preset test in Unity on Mac4, approve the visual target, then author `LumeFlowParticles.vfx` only after the overlay version proves the look.