NACE Professional Development recognizes the unique needs of career services and university relations and recruiting professionals. Our comprehensive training programs support growth at every stage of your career.

As a trusted source of high-quality, evidence-based education, NACE Professional Development offers an extensive selection of SHRM, NBCC, CCE, and HRCI-approved courses to help professionals meet continuing education requirements. Each year, we introduce new topics developed in collaboration with experts from the field to ensure our programs remain relevant and impactful.

If all of the property elements on a blank node element have string literal values with the same in-scope xml:lang value (if present) and each of these property elements appears at most once and there is at most one rdf:type property element with a RDF URI reference object node, these can be abbreviated by moving them to be property attributes on the containing property element which is made an empty element.

the forward sensitivity equations can still be derived by formal differentiation under the stochastic integral, yielding an augmented linear SDE driven by the same Wiener process. Automatic differentiation through a fixed-step Euler–Maruyama or Milstein discretisation recovers exactly that augmented system (up to the usual floating-point and common-subexpression differences). Adaptive or higher-order schemes, however, introduce extra randomness through the step-size controller or through Brownian-bridge interpolants; pure discrete AD therefore differentiates those random decisions as well. The resulting gradient estimator remains unbiased for the discrete path measure, but is generally biased (and higher-variance) for the continuous Itô measure that the modeller usually intends.

Continuous adjoint methods for SDEs avoid differentiating the adaptive machinery by working with the Kolmogorov backward equation or with a stochastic adjoint SDE that is integrated backwards along a stored forward path. Because the diffusion term produces a second-order operator, the continuous adjoint is a linear parabolic PDE (or its Monte-Carlo counterpart). In practice one therefore still needs automatic differentiation—now to form the vector-Jacobian products that appear inside the adjoint drift and diffusion—exactly as in the deterministic case. The performance gap between a naïve finite-difference Jacobian and a reverse-mode VJP is even larger for SDEs, because the diffusion matrix is typically dense.

Jump processes add a further discrete layer. For a pure-jump Markov process the intensity \(\lambda(u,p)\) and the jump map \(\phi(u,p)\) are themselves differentiable. Differentiating the stochastic simulation algorithm (SSA) path-by-path produces a discontinuous derivative that jumps whenever a reaction fires; the continuous adjoint, by contrast, evolves a measure-valued process between jumps and applies an instantaneous jump condition derived from the infinitesimal generator. Once again the generator itself is most efficiently evaluated by automatic differentiation of the intensity and stoichiometry functions.

Higher-Order Sensitivities and Hessian-Vector Products

Second- and higher-order derivatives of a solver output with respect to parameters are needed for Newton-type optimisation, Laplace approximations, and uncertainty quantification. The naïve approach—nesting forward-mode AD inside reverse-mode AD, or vice versa—works, but quickly becomes memory- and compute-intensive. Continuous second-order adjoints exist: one solves a further linearised equation whose right-hand side contains the second derivatives of \(f\) contracted with the first-order sensitivities. Those second-derivative tensors never need to be formed explicitly; they appear only as directional derivatives that a nested AD tool can evaluate on the fly. In the SciML stack this is realised by composing ForwardDiff (or Enzyme) with the continuous adjoint of OrdinaryDiffEq, so that the outer reverse pass never materialises a full Hessian.

For implicit solvers the same idea reappears via the implicit-function theorem at second order. If \(F(x^*,p)=0\), the Hessian of \(x^*\) satisfies a linear equation whose coefficients are the first and second derivatives of \(F\) evaluated at the converged point. Differentiating the Newton iteration itself would require propagating second-order information through every step; the IFT route evaluates those derivatives only once, at the final iterate.

In every regime the fastest practical implementations still rely on automatic differentiation—either to differentiate the solver steps directly or to supply the directional derivatives that appear inside analytically derived sensitivity equations. The art is knowing which of those two roles AD should play for the problem at hand.

A Note on Stability versus Performance (Revisited)

As already hinted in the continuous-versus-discrete discussion, there remains a persistent trade-off between numerical stability and raw speed. Discrete adjoints obtained by reverse-mode AD through the actual solver steps inherit the stability properties of the underlying discretisation; continuous adjoints can sometimes introduce additional stiffness or spurious oscillations, especially when the forward problem is only marginally stable. Conversely, continuous formulations often allow larger steps and cheaper linear algebra. The literature contains examples in both directions; the practical recommendation is therefore to prototype both approaches on a representative subset of the parameter space and to monitor the condition number of the resulting sensitivity equations.

This observation extends naturally to the implicit-function setting: the derivative obtained from the converged residual is only as accurate as the residual itself. Tightening the nonlinear tolerance (or using a higher-order corrector) improves gradient fidelity at a cost that is usually far smaller than differentiating every intermediate Newton iterate.

There is still a great deal of active research at the intersection of stochastic simulation, higher-order adjoints, and structure-exploiting AD. If any of the above regimes matches a problem you are currently facing, the SciML organisation is always interested in concrete test cases that expose new performance or stability edge cases.