Foundations Models for PDEs

Diagram of a Fluid Foundation Model

Creating machine learning surrogate models that capture the dynamics of physical systems is a field that has seen rapid development in recent years. However, creating these models from scratch are both time consuming and data intensive. Foundation models for PDEs that can be finetuned to specific systems of interest offer a solution to this problem.

Most physical systems of interests share common physics between them. The goal is to create a model that is trained on multiple physics from diverse datasets, that is able to capture these common features and understand behavior of these systems across different scales. While efforts have started in this direction 1,2 , there are many questions to be explored regarding choice of model architecture, creation of larger and more diverse datasets, training strategies, and the effects of boundary conditions and geometries.

We can currently working on building a Vision Transformer that generalizes across the parameter space of a given PDE. Expect to see the paper on ArXiv soon.

The Team

Amal S Sebastian
Ph.D. Student, Aerospace Engineering
University of Michigan
Bharath Ramsundar
Founder and CEO
Deep Forest Sciences