import os
os.cpu_count()12Neural ODEs
Neural ODE, introduced in (Chen et al. 2019), represents a class of neural networks capable of performing continuous dynamics modelling. Based on the idea that resnets and recurrent models operate with transformations in the hidden state \[h_{t+1}= h_t + f(h_t, \theta_t), \; \text{as} \; \Delta t \rightarrow 0,\] the neural network parameterises the continous dynamics of an ODE \[\frac{dh(t)}{dt} = f(h(t), t, \theta)\].
The key contributions from this paper are:
1. Modelling continous dynamics
2. Obeying an Ordinary Differential Equation (ODE)
3. Using the adjoint-method to scale to deeper and more complex networks without memory constraints. Before we dive into the details of this work. Lets take a look at what it means to have a neural network to obey an ODE.
ODEs are solved as a systems of linear equations Here is a Python code cell:
References
Chen, Ricky T. Q., Yulia Rubanova, Jesse Bettencourt, and David Duvenaud. 2019. “Neural Ordinary Differential Equations.” https://arxiv.org/abs/1806.07366.