RESEARCH PAPER TECHNICAL LATEX

We present a mathematically rigorous framework for multi-modal sensor fusion in real-time embodied interaction systems. Our approach, \emph{Recursive Polymodal Synthesis} (RPS), fuses heterogeneous modalities with disparate sampling rates and noise statistics into a coherent latent representation suitable for generative control. The core mechanism is a proximal fixed-point iteration using spectral-norm-constrained relational operators, yielding contraction guarantees and a unique fixed point. We prove geometric convergence in $\mathcal{O}(\log(1/\epsilon))$ iterations to $\epsilon$-accuracy when $\alpha\lVert T\rVert_2<1$. The system uses modality encoders $\{E_m\}_{m=1}^M$, linear translators $\{T_m\}_{m=1}^M$ with $\lVert T_m\rVert_2\le \sigma_{\max}<1$, and an update $\Pprox_\alpha(\vz)=(1-\alpha)\Emap(\vx)+\alpha\,T(\vz)$. On synthetic data, the approach attains $99.94\%$ cross-modal coherence, validation loss $1.93\times 10^{-4}$, and $15$--$40$\,ms CPU latency. Ablations confirm spectral constraints are necessary and reveal low effective rank structure in learned translators. RPS provides a principled, low-latency alternative to attention-based fusion with explicit robustness to missing modalities and tight stability bounds, enabling live performance, human--robot interaction, and adaptive interfaces.