WELCOME TO THE SciFM SUMMER SCHOOL


MICDE recently organized the SciFM conference and had an incredible reception for the event. Based on the feedback from that meeting and further discussions with National Lab and academic partners, we organized the SciFM Summer School 2024. We wish for this to be a recurring event hosted at different locations across the country. The first instance was run in parallel at the University of Michigan (UM) and at Argonne National Laboratory (ANL). The summer school involved:

  • Developing technical plans for specific instances of SciFMs;
  • A hackathon to jumpstart the construction of SciFMs;
  • Furthering a vision for a national SciFM ecosystem.

Lectures

Video recordings of the lectures are now available on YouTube.

Tutorial Resources

We have made our course content available on GitHub at scifm/summer-school-2024, licensed under the Apache 2.0 license. You willll find pre-worked examples covering topics such as tokenization and neural scaling laws, along with demonstrations of vision transformers and diffusion models tailored for scientific applications.

Core Organizers

Venkat Viswanathan
Associate Professor of Aerospace & Mechanical Engineering
University of Michigan
Karthik Duraisamy
Karthik Duraisamy
Director of MICDE
University of Michigan
Arvind Ramanathan
Computational Science Leader 1
Argonne National Laboratory

Hackathon Organizers

Anima Anandkumar
Bren Professor of Computing and Mathematical Sciences
California Institute of Technology
Michael Mahoney
Group Lead Machine Learning and Analytics
Lawrence Berkeley National Laboratory

Participants explored innovative applications of Foundation Model in four application domains:

  1. Materials -

    Foundation Models for material science have been used for a range of material property prediction, retrosysnthesis and structure search tasks.

  2. Biology -

    In Biology, Foundation Models have enable numerous breakthroughs including protein structure prediction and the classfication of emergent variants of viruses.

  3. Computational Science Agents -

    A Computational Science Foundation Model capable of designing and orchestrating computational research campaigns will enable scientists to focus on domain science through the creative specification of hypotheses and greatly accelerate scientific discovery.

  4. Partial Differential Equations -

    Foundation Models may be the route to enabling data efficient, general purpose PDE solvers.

Download the schedule here.
Week 1
Monday, July 8
10:00 am - Welcome & Opening Remarks - Karthik Duraisamy (UM)
10:10 am - Summer School Goals - Venkat Viswanathan (UM)
10:30 am - Keynote: Foundation Models for Science - Ian Foster (ANL)
12:00 pm - Open Discussion
1:30 pm - Foundation Models for Materials Discovery - Jan Janssen (Max Planck Inst.)
2:15 pm - Materials Foundation Models - Philippe Schwaller (EPFL)
3:00 pm - BREAK
3:15 pm - Foundation Models for Drug Discovery - Bharath Ramsundar (Deep Forest Sciences)
4:00 pm - Foundation Models for Climate - Aditya Grover (UCLA)
FoundSci Session
10:00 am - FoundSci Program Vision - Erica Briscoe (DARPA)
10:20 am - Jia Deng (Princeton)
10:40 am - Wei Wang (UCLA)
11:00 am - Robotics - Boyuan Chen (Duke)
11:20 am - Creativity - Lav Varshney (UIUC)
11:40 am - Symmetries - Rose Yu (UCSD)
Visioning Session (Closed Group)
1:30 pm - Existing Structures & Institutes (Discussion)
2:00 pm - Vision & SciFM Institute: Proposed Structure
3:00 pm - Working Breakout Session: Prompt - Produce a blueprint for SciFM Institute (resources, personnel, structure)
Tutorial 1
9:00 am - Climate Foundation Models - Paris Perdikaris (Microsoft/UPenn)
11:00 am - PDE Foundation Models - Maximilian Herde (ETH)
1:30 pm Hackathon Details & Logistics
Tutorial 2
2:00 pm - Data & Tokenization
Sequence data: Omics + Molecular - ANL Team
Tokenization - UMich & ANL Team
Discussion of Practical Issues
Tutorial 3
9:30 am - Training Models
Basics of sequence transformer
Basics of vision transformer
Neural operators & other PDE architectures - CalTech Team
1:30 pm - Diffusion Models - Karthik Duraisamy (UMich)
3:00 pm - Scaling Laws - Alexius Wadell (UMich)
Overview of Scaling Laws
Scaling sequence models & compute optimal scaling
Vision scaling laws
Scaling laws of FNOs and other PDE models
Tutorial 4
9:30 am - Retrieval Augmented Generation (RAG) - Ozan Gokdemir (ANL)
11:00 am - Agents: Part 1 Biology - ANL Team
1:30 pm - Agents: Part 2 Chemistry - ANL Team
Integrating external tooling and experimental data observations
Fine-tuning + Preference
3:00 pm - State-of-the-art in language models and agents in chemistry - Christopher Collison
4:00 pm - Hackathon - Final team selection & goal setting
Week 2
Monday, July 15
Hackathon
Hackathon
Hackathon
Tutorial 5 - AI Accelerators
10:30 am - Welcome
10:45 am - Cerebras
11:15 am - Sambanova
11:45 am - Break
12:00 pm - Graphcore
12:30 pm - Groq
01:00 pm - Lunch
02:00 pm - Nvidia
02:30 pm - Hands On
04:00 pm - Break
04:15 pm - Open Discussion, Q/A
Hackathon
Tutorial 6
Federated Learning - Ravi Madduri (ANL)
Hackathon
Week 3
Monday, July 22
Hackathon
Tuesday, July 23
Hackathon
Wednesday, July 24
Hackathon
Hackathon Team Presentations (industry present)
Materials
Biology
Agents
PDEs
Outlook for AI-assisted science in non-research settings
09:30 am - Open questions for Foundation Models for Science
11:00 am - Panel: Security, Equity, Safety
01:00 pm - Ecosystem Gathering (University, Lab, DoD, Venture, Philanthropy)
02:30 pm - Next steps and closing discussion