Beyond -- Anticipation Geometry for NUMU FARE

Beyond -- Anticipation Geometry for NUMU FARE

What Beyond Is

Beyond is a NUMU FARE package (`numu-beyond`) that uses anticipation geometry to orchestrate three AI paradigms through a single geometric coordination signal. Instead of each paradigm implementing its own convergence detection, retry logic, and stall recovery, Beyond provides a universal orchestration loop driven by four mathematical scalars computed over the trajectory of bus events.

The three paradigms:
- OmniFlow: Ensemble simulation with counterfactual probing
- Draft-and-Prune: LLM generation with deterministic verification
- Prompt Optimization: Search over prompt configurations for quality convergence

The key insight: all three follow the same geometric trajectory. They start with high uncertainty and low commitment (exploring), build transition pressure as they narrow (converging), and end with high commitment and low uncertainty (locked). The anticipation scalars make this universal trajectory visible and actionable.

          +-----------+     commitment > 0.5      +------------+     commitment > 0.7
          | EXPLORING |  ---------------------->  | COMMITTING |  ------------------>  +--------+
          |           |  <----------------------  |            |                        | LOCKED |
          +-----------+   uncertainty > 0.6       +------------+                        +--------+
               ^  |            ^                      |    ^                                |
               |  |            |   transition          |    |                                |
               |  +--- STALLED |   pressure > 0.5      v    |                                |
               |       (stuck) |                  +---------------+    transition              |
               |               +------------------| TRANSITIONING |    pressure > 0.5         |
               |                                  +---------------+ <--------------------------+
               |                                        |
               +--- uncertainty > 0.6, recovery > 0.5 --+

The Four Scalars

These are computed from the trajectory of embedded bus events for each running process.

The regime classifier maps scalar combinations to five states:

Architecture

                         NUMU FARE Daemon (Mac1, :7890)
                         ==============================

  +---------+    +----------+    +-----------+    +---------+    +--------+
  | Discord |    | Sentinel |    |  Cortex   |    |  KARL   |    | Beyond |  <-- NEW
  +---------+    +----------+    +-----------+    +---------+    +--------+
       |              |               |                |              |
       +------+-------+-------+-------+--------+------+------+-------+
              |               |                |              |
         +----v----+     +----v----+      +----v----+    +----v----+
         | numu-bus|     | numu-   |      | numu-   |    | numu-   |
         | (:7890) |     | registry|      | router  |    | beyond  |
         +---------+     +---------+      +---------+    +---------+
                                                              |
                              +-------------------------------+
                              |
                    +---------+---------+
                    |  BeyondProcess    |
                    |  Manager          |
                    |                   |
                    |  +------------+   |
                    |  | Scalar     |   |
                    |  | Engine     |   |     +-- OmniFlow Adapter
                    |  +-----+------+   |     |
                    |        |          +-----+-- DraftPrune Adapter
                    |  +-----v------+   |     |
                    |  | Embedder   |   |     +-- PromptOpt Adapter
                    |  | (Tier 1/2) |   |
                    |  +------------+   |
                    |                   |
                    |  Meta-Trajectory  |
                    |  (self-monitor)   |
                    +-------------------+

Package Structure

numu-beyond/
  src/
    index.ts           # Package entry, NUMUBeyond class, wire()
    scalars.ts         # BeyondScalarEngine (TS port of generalized_anticipation.py)
    embedder.ts        # BeyondEmbedder (Tier 1 hash / Tier 2 Gemini / Tier 3 passthrough)
    process.ts         # BeyondProcessManager, BeyondProcess, state machine
    paradigm.ts        # BeyondParadigm interface
    adapters/
      omniflow.ts      # OmniFlow paradigm adapter
      draftprune.ts    # Draft-and-Prune paradigm adapter
      promptopt.ts     # Prompt Optimization paradigm adapter
  test/
    golden/            # 50 test trajectories from Python reference
    beyond-scalars.test.ts
    beyond-process.test.ts
    beyond-embedder.test.ts

1. NUMU FARE Topics Map to Anticipation Scalars

New Bus Events

`beyond.trajectory` (local-only by default, high frequency)
Published every N events or T seconds for each active process. Carries the current scalar snapshot and state machine position.

`beyond.transition` (broadcast, on state change)
Published when the geometric state machine transitions between states. Includes from/to states, trigger description, and scalar values at transition time.

`beyond.intervention` (broadcast, on paradigm action)
Published when a paradigm adapter takes corrective action. Includes intervention type (counterfactual, feedback_inject, expand, restart, escalate) and paradigm-specific payload.

`beyond.convergence` (broadcast, on process completion)
Published when a process reaches LOCKED state. Carries the final result, scalar history, total events processed, and duration.

Mapping Existing Events to Trajectories

Beyond subscribes to existing bus events and routes them to active processes based on event type and process configuration:

2. The Universal Orchestration Loop

                    +-------+
                    | SENSE |  Subscribe to bus events. Embed each event.
                    +---+---+  Feed embedding to scalar engine.
                        |
                        v
                    +--------+
                    | DECIDE |  Compute scalars. Run regime classifier.
                    +---+----+  Check state machine transitions.
                        |
                        v
                    +-------+
                    |  ACT  |  If state changed: publish beyond.transition
                    +---+---+  If STALLED: call adapter.onStall()
                        |      If LOCKED: call adapter.onConverge()
                        v      If pressure dropping: call adapter.onIntervene()
                    +--------+
                    | RESENSE|  The intervention/convergence events themselves
                    +---+----+  become new trajectory points, closing the loop.
                        |
                        +-----> (back to SENSE)

The loop runs for every bus event that matches a process's subscription filter. There is no polling interval -- it is purely event-driven. The `beyond.trajectory` snapshot is published at a configurable interval (default: every 10 events or every 10 seconds) to avoid flooding.

Concrete Example: Draft-and-Prune Building an iOS App

Event 1:  task.dispatch (generate app code)          -> EXPLORING  (u=0.8, c=0.2)
Event 2:  task.completed (code generated)             -> EXPLORING  (u=0.7, c=0.3)
Event 3:  build.failed (compilation error)            -> EXPLORING  (u=0.6, c=0.2)
Event 4:  task.dispatch (regenerate with error feedback) -> COMMITTING (u=0.5, c=0.5)
Event 5:  task.completed (revised code)               -> COMMITTING (u=0.4, c=0.6)
Event 6:  build.completed (compilation success!)      -> LOCKED     (u=0.2, c=0.8)
   --> beyond.convergence published, result accepted

If instead the build fails 5 times in a row:

Events 3-12: build.failed repeatedly               -> STALLED   (u=0.1, c=0.1)
   --> beyond.intervention published, adapter.onStall() called
   --> Adapter escalates to human or switches verification strategy

3. How Each Paradigm Plugs In

OmniFlow (Ensemble Simulation)

What it does: Dispatches N parallel tasks with varying parameters. Monitors the spread of results. If spread is high (high uncertainty), spawns counterfactual probes with modified parameters. Loops until results cluster tightly (low uncertainty, high commitment).

Subscribed events: `task.dispatch`, `task.completed`, `task.failed`

Embedding strategy: Tier 1 structural hash by default. Key dimensions encode: output length variance, error/success ratio, parameter diversity.

Intervention logic:
- `onIntervene()`: When transition_pressure drops below -0.3 (diverging instead of converging), dispatch N/2 additional tasks with parameters interpolated between the best and worst results.
- `onConverge()`: When uncertainty < 0.2, compute centroid of all completed task outputs. Return as result.
- `onStall()`: If 3+ rounds of probes show no uncertainty reduction, escalate or increase ensemble size by 2x.

NUMU integration: Uses `task.dispatch` with `machineTarget` to spread ensemble across mesh machines. Uses `task.completed` outputs as the trajectory data.

Draft-and-Prune (Neuro-Symbolic Verification)

What it does: Dispatches a generation task (LLM writes code/translation/proof). Routes output to a deterministic verifier (compiler, type checker, SAT solver). If verification fails, regenerates with accumulated error feedback. Loops until verification passes.

Subscribed events: `task.completed`, `task.failed`, `build.completed`, `build.failed`

Embedding strategy: Tier 1 structural hash. Key dimensions encode: attempt number (normalized), error type hash, success/failure binary, feedback injection count.

Intervention logic:
- `onIntervene()`: When recovery_margin drops below 0.3 (deep into a failing branch with no easy backtrack), inject all accumulated error messages into the next generation prompt as negative examples.
- `onConverge()`: When build.completed fires and commitment > 0.7, accept the generated artifact.
- `onStall()`: If rejection-regeneration exceeds 5 cycles (configurable), switch verification strategy (e.g., from full build to syntax check only) or escalate to human.

NUMU integration: Uses existing `task.dispatch` for generation and the mesh build pipeline (`build.completed`/`build.failed`) for verification. The adapter manages the generate-verify-feedback loop automatically.

Prompt Optimization (Quality Convergence)

What it does: Maintains a population of prompt variants. Evaluates each against a quality metric (LLM-as-judge score, task success rate, output parsability). Selects the best, mutates to create new variants, and iterates until quality plateaus.

Subscribed events: Custom `beyond.prompt.scored`, `beyond.prompt.mutated`

Embedding strategy: Tier 2 Gemini embedding recommended (prompt quality is semantic, structural hash may miss it). Embed prompt structure features: section count, instruction ordering, few-shot example count, CoT placement.

Intervention logic:
- `onIntervene()`: When transition_pressure spikes (rapid quality oscillation), reduce mutation rate and increase elitism (keep more top variants).
- `onConverge()`: When quality score variance < 0.05 across top 3 variants for 5+ iterations, freeze the best prompt.
- `onStall()`: If quality is below threshold and not improving, reseed the population from a different base prompt or add few-shot examples from a different domain.

NUMU integration: Prompt variants are dispatched as `task.dispatch` with `forgeTemp: "ember"` (fast, cheap evaluation). Quality scores are computed by the adapter and published as `beyond.prompt.scored`.

4. Anticipation Geometry Replaces Domain-Specific Convergence

The traditional approach to orchestrating these paradigms requires domain experts to define:
- OmniFlow: "convergence = standard deviation of outputs < 0.1"
- Draft-and-Prune: "convergence = verifier returns true"
- Prompt Optimization: "convergence = quality metric plateaus for N iterations"

Beyond replaces all of these with a single geometric criterion:

Convergence = commitment > 0.7 AND uncertainty < 0.3 AND recovery < 0.4

This works because the scalars are domain-agnostic. They measure the GEOMETRY of the event trajectory, not the SEMANTICS. When OmniFlow results cluster tightly, the trajectory vectors cluster tightly, commitment rises, uncertainty falls. When Draft-and-Prune's verifier passes after several failures, the trajectory enters a new region and stays there, commitment rises. When Prompt Optimization's quality plateaus, the trajectory stops moving, commitment rises.

The domain-specific thresholds (output std < 0.1, verifier returns true) are still available as paradigm adapter logic, but they're secondary to the geometric signal. The adapters use them to interpret WHAT converged, while the geometry detects THAT convergence happened.

Advantage: A new paradigm (e.g., "Evolutionary Architecture Search") can plug into Beyond with zero convergence logic -- just define `subscribedEvents()` and `embedEvent()`, and the geometric state machine handles the rest.

5. Auto-Research: Beyond Monitors Itself

Beyond maintains a meta-trajectory computed from the aggregate scalars of all active processes.

Process A (OmniFlow):    c=0.6, u=0.4, p=+0.2  -> COMMITTING
Process B (DraftPrune):  c=0.1, u=0.1, p=-0.1  -> STALLED
Process C (PromptOpt):   c=0.8, u=0.2, p=+0.0  -> LOCKED

Meta-trajectory point: c=0.50, u=0.23, p=+0.03
Meta-state: COMMITTING (overall making progress)

But Process B is STALLED. Meta-system detects:
  - Any subprocess in STALLED for > stallThreshold events
  - Publishes beyond.intervention { type: "meta-diverge", processId: B }
  - Process B's adapter.onStall() is called

The meta-trajectory is NOT recursive. It is a single additional scalar engine that aggregates child process snapshots. It cannot observe itself. This avoids the halting-problem-adjacent issues of recursive self-monitoring while still providing system-level health detection.

Auto-research in practice: When Beyond detects that its OWN orchestration is stalling (e.g., 3 processes running but none converging, meta-trajectory enters STALLED), it can:
1. Identify the most-stuck subprocess
2. Trigger a paradigm-appropriate intervention
3. Log the intervention + outcome to KARL as training data
4. Future trajectory analyses benefit from KARL's learned intervention effectiveness

6. Integration with Existing Infrastructure

Graph Kernel (GK, :8001 on cloud-vm)

Used by Draft-and-Prune when verification requires knowledge graph validation. The adapter calls GK's edge-existence API (same as `kg_reward.py`) to verify reasoning paths. This is adapter-specific, not a core Beyond feature.

The KG-path reward function provides a complementary signal to anticipation scalars:
- Scalars answer: "Where is this trajectory going?" (dynamics)
- KG rewards answer: "Is this path valid?" (correctness)
- Combined: early detection of processes heading toward invalid territory

KARL (Trajectory Intelligence, :9200 on Mac5)

Every `beyond.convergence` and `beyond.intervention` event is logged as a KARL trajectory entry with:
- Domain: `beyond`
- Skill: `{processType}` (omniflow, draftprune, promptopt)
- Reward: scalar-derived score (commitment_final * (1 - uncertainty_final))
- Advantage: how much faster this process converged compared to baseline

Over time, KARL learns which intervention strategies work for which scalar patterns, enabling the system to get better at self-correction.

Mesh Compute (Mac1-5, cloud-vm)

OmniFlow's ensemble dispatch uses existing `task.dispatch` with `machineTarget` to spread work across the mesh. Beyond doesn't manage machines -- it manages trajectories. Machine allocation is handled by the existing dispatch/fleet layer.

Nexus Portal (:3001 on cloud-vm)

A new `/beyond` dashboard page displays:
- Active processes with state machine visualization
- Real-time scalar time series (commitment, uncertainty, pressure, recovery)
- State transition timeline
- Intervention log
- Meta-trajectory health indicator

Prefect Flows (:4200 on cloud-vm)

Beyond processes can be triggered by Prefect flows. A flow calls the NUMU HTTP API to start a Beyond process, then polls for `beyond.convergence` or `beyond.intervention` events. This enables batch orchestration (e.g., "run 10 prompt optimization experiments overnight").

7. Implementation Plan

### Phase 1: Scalar Engine (Days 1-3)
Build the TypeScript port of anticipation geometry. Validate against 50 golden-file test trajectories generated from the Python reference. This is the foundation -- if the math is wrong, nothing else works.

Deliverables: `numu-beyond/src/scalars.ts`, `test/golden/*.json`, `test/beyond-scalars.test.ts`

### Phase 2: Embedding + Process Manager (Days 4-6)
Build the event embedder (structural hash + optional Gemini), process lifecycle, and geometric state machine. Validate state transitions against synthetic event streams.

Deliverables: `numu-beyond/src/embedder.ts`, `numu-beyond/src/process.ts`, `test/beyond-process.test.ts`

### Phase 3: Bus Integration (Day 7)
Wire Beyond into the NUMU daemon. Add TypeBox schemas. Integrate with daemon startup. Test with live bus events.

Deliverables: Schema additions to `numu-bus/src/schema.ts`, daemon.ts integration, HTTP endpoints

### Phase 4: Paradigm Adapters (Days 8-10)
Build the three paradigm adapters. Test Draft-and-Prune with a real iOS build cycle. Validate that the geometric state machine correctly detects convergence/stall/intervention points.

Deliverables: `numu-beyond/src/adapters/*.ts`, end-to-end test

### Phase 5: Meta-Trajectory + KARL (Days 11-13)
Build self-monitoring. Wire KARL integration. Deploy Nexus dashboard page. The system is complete.

Deliverables: Meta-trajectory in process.ts, KARL logging, `/beyond` dashboard

### Kill Criteria
- Day 5: If TS scalar engine diverges from Python by > 5
- Day 8: If structural hash embeddings produce < 50
- Day 14: If no real process has run to completion, Beyond lacks a use case. Pause.

---

Appendix: Configuration Reference

# [home-path] additions

[beyond]
enabled = true
window_size = 100             # trajectory ring buffer length
publish_interval_ms = 10000   # ms between beyond.trajectory broadcasts
broadcast_trajectory = false  # false = publishLocal, true = publish (broadcast)
meta_trajectory = true        # enable self-monitoring meta-level

[beyond.thresholds]
exploring_to_committing_commitment = 0.5
committing_to_locked_commitment = 0.7
committing_to_locked_uncertainty = 0.3
committing_to_locked_recovery = 0.4
transitioning_pressure = 0.5
exploring_uncertainty = 0.6
exploring_recovery = 0.5
stall_events = 10
stall_commitment_max = 0.3
stall_uncertainty_max = 0.3
stall_pressure_max = 0.1

[beyond.embedding]
default_tier = 1                     # 1=hash, 2=gemini, 3=passthrough
hash_dimensions = 32
gemini_model = "gemini-2.0-flash"
gemini_cache_ttl_ms = 300000
gemini_rate_limit_per_min = 60

[beyond.paradigms.omniflow]
embedding_tier = 1
ensemble_size = 4
counterfactual_probe_count = 2
max_rounds = 10

[beyond.paradigms.draftprune]
embedding_tier = 1
max_cycles = 5
feedback_accumulation = true

[beyond.paradigms.promptopt]
embedding_tier = 2
population_size = 8
elitism_count = 2
mutation_rate = 0.3
max_iterations = 20