flowchart LR A(Encoder) --> B(Processor) B --> C(Decoder)
Geometry for Neural-PDE surrogates
Most neural PDE solvers within the research ecosystem are built looking at regular geometry with structured uniform grids. Though this conjures up a great starting point to look at solving complex phenomena, it is only mildly representative of the computational physics cases that we are interested in practice. Regular grids have been a beneficial starting point as it is easy to port the wealth of research done within computer vision with its clear 3D vioxel structure to more scientific applications. Though there are certain areas where regular geometry finds application, before we can move this research to product, we will need to address for irregular unstructured grids at vast scales (~ millions of cells).
This has been a large area of interest in the recent years and the work at modelling spatio-temporal data across irregular geometries can be split into the following categories:
- Graph-based Methods : GNNs, GCN, GINOs. Involves message passing to emphasise the geometric structure. (Li et al. 2023), (Li et al. 2020)
- Point Cloud Approaches : Coordinate based-MLPs, Neural Fields, INRs. No emphasis on graph structure apart from coordinates and perhaps the associated SDF.
- Kernel-based Methods : GPs. Function fitting and infering the field values at specific geometric positions.
The table below outlines a structured comparison table of machine learning methods for handling irregular geometries:
| Method | Advantages | Limitations |
|---|---|---|
| Graph based Methods | - Natural handling of irregular spatial relationships through message passing - Excellent capture of local geometric features - Works with varying numbers of spatial points - Maintains permutation invariance - Easy incorporation of additional features at spatial locations |
- Struggles with global geometric patterns - Computationally expensive for dense graphs - Performance depends heavily on graph construction - Training stability issues in deep architectures - Message passing can be inefficient for long-range dependencies |
| Point Cloud Approaches | - Direct processing of unstructured data without connectivity information - Effective for 3D data representation - Handles varying point densities - Natural permutation invariance - Flexible with varying point cloud sizes |
- Difficulty capturing fine geometric details - Sensitive to point density variations - Requires careful input data preprocessing - Poor scaling of memory requirements with point count - May miss local spatial relationships |
| Kernel-Based Methods | - Natural uncertainty quantification - Effective handling of irregular sampling - Works well with limited data - Incorporates prior knowledge via kernel design - Provides smooth interpolation |
- Poor scaling with dataset size - Sensitive to kernel choice - Hyperparameter selection challenges - Struggles with discontinuities - Difficulty capturing sharp geometric features |
But for large scale neural PDE models modelling complex spatio-temporal domain with large context lengths, the modelling task is split into a encoder-processor-decoder architecture.
The layout breaks down the modelling task into three parts. The encoder works within the spatial domain to deconstruct the initial conditions of the fields and the inherent geometry of the problem into a latent, more structured space. The processor learns to map the temporal evolution of the PDE within that klatent space. The choice of the processor is often taken to be NN architecture that is quick to evaluate and could be setup as a Neural ODE. The decoder maps the final solution from the latent space to the actual unstructured grid that we are interested in.
These seems to be adjacent to the kind of structure that we are looking at with a division of power and responsibilities, with specific networks and models learning differnet parts of the task.
References
Li, Zongyi, Nikola Borislavov Kovachki, Chris Choy, Boyi Li, Jean Kossaifi, Shourya Prakash Otta, Mohammad Amin Nabian, et al. 2023. “Geometry-Informed Neural Operator for Large-Scale 3D PDEs.” In Thirty-Seventh Conference on Neural Information Processing Systems. https://openreview.net/forum?id=86dXbqT5Ua.
Li, Zongyi, Nikola Kovachki, Kamyar Azizzadenesheli, Burigede Liu, Andrew Stuart, Kaushik Bhattacharya, and Anima Anandkumar. 2020. “Multipole Graph Neural Operator for Parametric Partial Differential Equations.” In Advances in Neural Information Processing Systems, edited by H. Larochelle, M. Ranzato, R. Hadsell, M. F. Balcan, and H. Lin, 33:6755–66. Curran Associates, Inc. https://proceedings.neurips.cc/paper_files/paper/2020/file/4b21cf96d4cf612f239a6c322b10c8fe-Paper.pdf.