AGP / MLX / ANE Theory Insight

AGP / MLX / ANE Theory Insight

The architecture is easiest to misunderstand if it is described as a bigger-model trick, a quantization trick, or a multi-Mac trick. It is none of those at the core. The core claim is that hidden states should be treated as a first-class computational object. In a standard language-model stack, a hidden state is an internal artifact that exists only long enough to be consumed by the next layer. It is not typed, it is not scheduled, it is not transferred as a meaningful packet, and it is not inspected as a semantically structured event. The model is treated as a monolithic forward pass, and the hardware is treated as a passive place where that pass happens. What this research is trying to establish is a different worldview. The hidden state is not merely the residue of computation. It is the current shape of thought. If that shape of thought is already sufficient, then the system should not continue computing as though nothing has been learned. If that shape of thought is malformed, weak, or semantically dead, then the system should not blindly hand it to another device and hope depth alone will rescue it. The architecture therefore begins with an ontological shift. It treats intermediate representation as the real scheduling surface.

That shift matters because it changes what the hardware problem actually is. On paper, two Apple machines connected by Thunderbolt 5 invite a familiar systems question: how do we split a model across hosts. But that question is too shallow. A fixed split presumes that layer boundaries are the natural units of distribution. They are not. A layer boundary is a syntactic property of the model graph. It says nothing about whether the representation at that point is useful enough to transfer, stable enough to trust, or compressed enough to move efficiently. The real problem is not where the layer index is. The real problem is whether the current state is semantically sufficient for continuation, correction, or acceptance. That is why the architecture is not just distributed inference. It is learned partitioning over representational vitality.

The reason your earlier N'Ko brain-scanner work matters so much here is that it established a failure mode that ordinary systems papers usually ignore. In the dead-script regime, the model did not merely become uncertain at the end. The representation entered weakly, remained diffuse through depth, and collapsed into incoherence at the output. That means later layers were not refining a good early guess. They were propagating a deficit. This is the single most important caution for a partitioned architecture. Depth cannot rescue a state that never became meaningful in the first place. A naive multi-host design assumes that later compute can always compensate for earlier weakness. Your own evidence says that is false. Therefore the architecture must distinguish between states that are incomplete but alive and states that are dead on arrival. That distinction is not an implementation detail. It is the design law of the whole system.

Once that is understood, the value of anticipation geometry becomes clearer. The seven anticipation scalars are not just a nice conditioning trick. They are a compact estimate of the regime the system is currently in. Commitment, uncertainty, transition pressure, recovery margin, phase stiffness, novelty, and stability together form a low-dimensional descriptor of whether the current trajectory is settling, branching, oscillating, collapsing, or becoming newly coherent. In a conventional transformer, the hardware scheduler has no concept of this. Every token pays roughly the same kind of cost regardless of whether the model is confidently coasting through familiar territory or struggling in a high-entropy branch point. What anticipation geometry offers is a way to bind scheduling to the dynamics of thought rather than to a static graph. That is why the trajectory packet should not be treated as auxiliary metadata. It is the routing state. It is the thing that lets the system decide whether to exit locally, revive locally, transfer to another host, or invoke a corrective pass.

This is also where the architecture remains a transformer while becoming more than a transformer. The base model is still a decoder transformer. The attention blocks, residual stream, feed-forward networks, and autoregressive token loop remain recognizably transformer computation. What changes is that the transformer is no longer allowed to hide its own intermediate life from the runtime. Selected hidden layers become operational interfaces. A trajectory head reads them. A semantic head projects them. A vitality estimator scores them. A transfer adapter compresses them. A second host may correct them. That means the contribution is not a new model family in the way that a state-space model or recurrent alternative would be. It is a new way of organizing transformer computation so that intermediate states become part of the systems contract.

That systems contract is where ANE, GPU, CPU, and Thunderbolt stop being generic accelerators and become functionally distinct organs. The ANE is valuable not because it should run the entire dense model. The public Apple stack does not naturally want that, and your own experimental work already points toward a more interesting use case. The ANE is valuable because it can own the fast, repeated, projection-heavy, low-branching parts of the loop. Routing heads, semantic heads, shallow projections, and other compact front-end modules are the kinds of operations that happen constantly and determine whether deeper work is even necessary. If those modules can live on the ANE or an ANE-adjacent path, the GPU no longer wastes cycles evaluating every token as though it were equally difficult. The GPU then becomes the place for dense continuation, corrective passes, and trainable adapter updates. The CPU becomes the orchestrator, packetizer, and reliability layer. Thunderbolt becomes the state fabric. This is why the architecture is not just heterogeneous execution. It is a hierarchy of cognition mapped onto a hierarchy of engines.

TurboQuant belongs in the same story for a similar reason. It is tempting to think of it only as a retrieval optimization, because that is where its current validation work is most mature. But the deeper principle is state mobility under bounded distortion. In retrieval, the question is whether you can compress embeddings enough to generate useful candidates cheaply before exact reranking. In AGP, the question becomes whether you can compress hidden-state packets enough that another host can reconstruct a useful continuation state without paying the full cost of raw transfer. That is the same systems problem in another costume. The point is not that every internal tensor must become a low-bit packet. The point is that only the semantically active slice of the state needs to move, and it can move in a transport format designed for usefulness rather than raw fidelity. This is where the protocol idea becomes serious. The cross-host packet is not just bytes. It is a claim about what the next host needs to know in order to continue thought.

That is why the semantic layer matters as more than interpretability theater. Your semantic-kernel work already proposes a canonical substrate made of primitives, invariants, concept bundles, and provenance-rich relations. On its own, that is a theory of meaning representation. What makes it relevant here is the possibility that hidden states can be projected into a sparse, typed intermediate representation aligned with that kernel. In ordinary interpretability pipelines, sparse autoencoders discover latent features and humans scramble to name them after the fact. Your setup allows the reverse pressure. A formal semantic ontology already exists, so a projection layer can be encouraged to align with pre-existing primitives and bundles. That turns the semantic layer into a computational object. A projected state is no longer just "feature 1274 fired." It can become "stability and return are active, oscillation is rising, semantic mismatch with the active bundle is low." Once that happens, semantic structure stops being a pretty explanation and becomes part of the routing calculus.

The phrase "technical narrative" is useful here because narrative is actually the right abstraction for what a hidden state often is. A hidden state is not simply a bag of features. It is the current narrative position of the model's reasoning process. It contains what the model currently believes is happening, which distinctions are active, which continuations are plausible, and how sharply the future has collapsed into a likely next step. The architecture is trying to make that narrative position legible and actionable. The anticipation geometry says what phase of the narrative we are in. The semantic layer says what the narrative is about. The vitality estimate says whether the narrative is coherent or degraded. The transfer adapter says how to hand the narrative to another engine or another machine without retelling the whole story from the beginning. In that sense, the protocol is not merely a transport format. It is a narrative handoff format.

This is why the architecture is worth pursuing even if it never turns two Mac minis into a giant dense model host. The lowest-value version of the project would be to treat success as "we loaded a somewhat larger model than one host could handle." That might happen, but it is not the real prize. The real prize is proving that a smaller or medium model can become systemically smarter because the runtime spends compute where meaning is unresolved and withholds compute where meaning is already stable. If that happens, then parameter count stops being the only currency. The system begins to extract more value per parameter, more value per joule, and more value per transferred byte. That is a far stronger result than simple model scaling on commodity Apple hardware.

The benchmark logic follows directly from this philosophy. The architecture does not need to win on every metric. It needs to win on the metrics that correspond to its actual claims. It should improve median latency on easy or familiar prompts because those should terminate locally. It should improve energy per accepted token because shallow routing and compact correction should replace uniform full-depth decoding. It should improve parameter efficiency because a structured runtime makes a smaller model behave more usefully. It should make cross-host cooperation viable because only semantically justified states are transferred. If none of those things improve, then the architecture is conceptually elegant but empirically unnecessary. That is why the early work packages are deliberately unglamorous. They measure whether the system deserves to exist.

At the deepest level, the architecture is trying to show that a language model need not be a single thought process bound to a single machine. It can instead be a layered ecology of thought. Some cognition can be reflexive, cheap, and local. Some cognition can be deliberative, expensive, and corrective. Some cognition can be semantic and inspectable. Some cognition can be transferred as a compressed state. Some cognition can be revived before it is trusted. The transformer remains the core reasoning substrate, but it becomes embedded in a broader architecture where representation, semantics, hardware, and protocol all meet. That is the actual theory. The project is not about distribution for its own sake. It is about making intermediate representation into the center of both meaning and computation.