Continuous-time Markov chains (CTMCs) describe discrete stochastic dynamics in physics, chemistry, and biology, but their categorical event choices block ordinary differentiation. In a new arXiv preprint, Jose M. G. Vilar and Leonor Saiz introduce the propensity straight-through (PST) estimator, which allows gradient-based learning through exact simulations of these systems.
PST keeps the forward simulation exact and, in the backward pass, differentiates the normalized reaction propensities that the simulator already computes. Because the state update is affine, the rule is exact for the one-step conditional-mean sensitivity; the authors derive the per-step discrepancy for composed nonlinear trajectories in closed form and bound it quadratically in the stoichiometric jump size.
Across reversible dimerization, a genetic oscillator, a 50-task repressilator suite, and experimental patch-clamp recordings, PST recovered parameters to within 0.06–1.7% and converged 2.4–3.0-fold faster than the Gumbel-Softmax straight-through estimator at matched computational cost. At deep-learning scale, PST trained all 203,796 parameters of a stochastic reaction network using exact discrete events from initialization, eliminating the need for reaction-selection temperature tuning.