System identification stops rerunning per record: one diffusion model holds state accuracy on records five times longer
Synopsis
Bayesian system identification infers a dynamical system's hidden states and parameters from measurements, and particle Markov chain Monte Carlo methods must be rerun for every new record and task. This work treats hidden-Markov-model system identification as a general Bayesian inverse problem and uses a conditional diffusion model trained once to approximate the joint posterior of states and parameters; at deployment, changing only the conditioning mask yields joint inference, state smoothing, parameter inference or missing-measurement reconstruction. On four benchmark systems, state accuracy holds on records five times longer than the training segments, while a dense-attention Simformer baseline's state error can grow more than tenfold.
Interpretation
A single trained conditional diffusion model returns the joint posterior of hidden states and parameters of a dynamical system and switches between inference tasks and record lengths without retraining. Particle Markov chain Monte Carlo methods must rerun the entire inference pipeline for every new record, task and missing-measurement pattern, with cost growing in state dimension and record length. On four benchmark systems, the same model performs joint inference conditioned on all measurements of a latent-variable system and reconstruction after consecutive measurements are removed, with the reconstruction band containing the true trajectory.
The model treats parameters, hidden states, measurements and inputs as separate token types and uses a Markov attention mask that keeps only the direct dependencies prescribed by the hidden-Markov-model factorization, so a state's direct neighborhood does not change as the record lengthens. Simformer distinguishes only parameters and data, gives hidden states no role of their own, and its dense attention lets every token attend to every other, with cost growing quadratically in record length. On lrw and dw, all combinations of Markov versus dense attention and rotary position encoding, Gaussian Fourier embedding or no time encoding are trained; state error stays nearly constant under the Markov mask and rises under dense attention as records lengthen.
Time enters only through differences between tokens, so the model trains on short segments and deploys on full records. An absolute-time Gaussian Fourier embedding must extrapolate to times outside the training distribution on records longer than the training trajectories. On lrw, as deployment length extends from the training length to five times it, the model's state error stays flat while the dense-attention Simformer baseline's state error grows more than tenfold.
Perspective
The demonstrated scope is four benchmark systems in which all states are measured with Gaussian noise of variance 1 and reference posteriors are available; the inference target is the joint posterior under a hidden-Markov state-space form, and deployment records extend to five times the training segment length.
The parameter posterior does not contract as records lengthen, because the model learns to aggregate parameter evidence only within its training windows; at the training length, even an eightfold training budget leaves parameter error about two to five times that of pmmh; the five-mask mixture and the random mask overlap in interval on joint inference, and whether the mixture helps the other tasks remains to be tested.
