Is this a recruiting attempt by Los Alamos? AI/ML for science as this broad field used to be known is interesting. Some five years ago there as a real craze where every STEM lab at my university was doing some form of ML project. I think by now people have learned what works and what doesn't. Climate models for example have been quite successful. Possibly the reason is that they learn directly from collected data, rather than trying to emulate the output of simulations. Attempts to build similiar models for fluid dynamics have been rather dismal. In general, big models and big data result in useful models, even if only because these models seem to be somehow interpolating based on similiar training data points. Trying to replace classical physics based models with ML models trained on simulation data does not seem to work. The model is only ever capable of emulating a physically plausible output when the input is close enough to the training data, and that too, only when the system isn't chaotic. For applications where you are generating a sample to be used in a downstream task, ML models trained on lots of data can be very useful. You only need a few lucky guesses, that you can verify downstream, to end up with some useful result.
In short, there is no magic to it. It's a useful tool that can be regarded as both a search algorithm and an optimization algorithm.
I find it interesting that fluid dynamics models struggle more than weather. Intuitively, my view of weather is that it looks a lot like fluid dynamics and seems quite chaotic. Is this wrong or is it just the vast amount of recorded weather data which helps?
edit: I realised just after posting that you only mentioned climate models. It was another poster who said there were successes using ML for meteorology.
I think an important question to ask is whether your scientific task is primarily one of interpolation, or one of extrapolation. LLMs appear to be excellent interpolators. They are bad at extrapolation.
I am already acquainted with them but to be honest, I am no longer in the field so I am not able to comment on latest developments. However, as of two years ago, the consistent result was that you could get models that reproduce really good physics for problems in the same physical regimes as the training data, but such models had poor generalizability, so depending on the use case, they weren't of much use. The only exception I know is FourCastNet, which is a weather model FNO from NVIDIA.
Yes. ML has advanced many fields related to modelling - meteorology, climate, molecular. Classification models have done much for genomics, particle physics, and other fields where experiments produce inhumane amounts of data.
DeepVariant, Enformer, ParticleNet, DeepTau, etc. are some well-known individual models that are advanced branches of science. And there are the very famous ones, like AlphaFold (Nobel in Chemistry 2024).
We need to think of AI not as a product (chats, agents, etc.), but as neural nets (AlexNet). Unfortunately, large companies are "chat-washing" these tremendously useful technologies.
ML is more of a bag of techniques that can be applied to many things than a pure domain. Of course you can study the properties of neural networks for their own sake but it’s more common as a means to an end.
Bit short on details other than "Let's see what LLMs can predict when we train them on various scientific data sets."
Certainly a good thing to try, but the article feels like a PR piece more than anything else, as it's not answering anything, just giving a short overview of a few things they're trying with no data on those things whatsoever.
It does fit in with the "Throw LLM spaghetti at a wall and see what sticks" trend these days though.
Should be possible to backtest by training LLMs on historic datasets and then probing them to see if they can re-discover things that were discovered after their training data cut-off. What sort of prompts could push them to make a breakthrough.
I think that’s an opportunity, not a problem. If prompt + hint generates a verifiable solution then you can build systems that propose hints, either randomly or by exploring a search space, and keep trying combinations until you hit on something that works.
exactly. hindsight bias makes it really hard to separate genuine inference from subtle prompt leakage. even framing the question can accidentally steer it toward the right answer. would be interesting to try with completely synthetic problems first just to test the method.
First, give it the abstract for a fresh paper that it couldn’t have been trained on, then see if it can come up with the same proofs to see if it can replicate the logic knowing the conclusion.
Second, you could give it all the papers cited in the intro and ask a series of leading questions like “based on this work, what new results can you derive”?
AlphaProof is among the most relevant methods here. And because it trains by self-play, instead of historical human data - it has a much better chances of being able to solve novel problems, or come up with solutions that humans have not. It did pretty good at the 2024 Olympiad. Will be interesting to see how 2025 goes.
Honestly, that's still far too much help in lots of cases.
Finding a set of papers, whose results can be combined in a reasonable amount of time to make a new interesting result is itself a hard problem. This is often a thing Professors do for PhD students -- give them a general area to research and some papers to start reading.
It's still a contribution, but so much easier than just asking "Hey, choose a set of papers from which you can derive new interesting results"
I feel like there are a few ways in which science could be advanced by AI models.
1. We have the sense that "science progresses one funeral at a time." An AI model could be used to recognize situations where a single viewpoint is getting a disproportionate amount of attention in the literature, and warn journals+funding agencies about any disconnects between attention and quality of work.
2. We have the sense that "It Ain’t What You Don’t Know That Gets You Into Trouble. It’s What You Know for Sure That Just Ain’t So" An AI model could identify the most high-profile results in the literature that have contradicting evidence and call for further, decisive study.
3. Interdisciplinary translation. There are many many cases of different branches of science re-discovering each other's work. I believe I read an article a little bit ago about an academic in a somewhat softer science publishing a paper proudly claiming the discovery of linear regression. Obviously not all cases are so egregious, but an AI could advance a discipline just by pointing out areas where that discipline is using outdated/inferior methods compared to the state of the art.
I think our creativity has not yet been duplicated in AI, so for maximum results, we need to pair AI with a human expert or a panel of human experts and innovate by committee. AI brings to the table vast memory, instant recall and most importantly, tired-less pursuit and the human element can provide creative guidance and prompt. The trick is in curating the BOK(body of knowledge) used to train GENERATIVE AI. I wonder what a curricula designed specifically for AI would look like?
Did… did this really long bio answer the question? I’m not sure what this is trying to do tbh, but it was interesting to see how these labs have been trying to incorporate AI.
I'm all for a renarrowing of the use of AI to no longer just cover any machine learning algorithm to be honest. It was a really annoying feature of the 2010s that any kind of machine learning application got the hype word du jour slapped on it.
Is this a recruiting attempt by Los Alamos? AI/ML for science as this broad field used to be known is interesting. Some five years ago there as a real craze where every STEM lab at my university was doing some form of ML project. I think by now people have learned what works and what doesn't. Climate models for example have been quite successful. Possibly the reason is that they learn directly from collected data, rather than trying to emulate the output of simulations. Attempts to build similiar models for fluid dynamics have been rather dismal. In general, big models and big data result in useful models, even if only because these models seem to be somehow interpolating based on similiar training data points. Trying to replace classical physics based models with ML models trained on simulation data does not seem to work. The model is only ever capable of emulating a physically plausible output when the input is close enough to the training data, and that too, only when the system isn't chaotic. For applications where you are generating a sample to be used in a downstream task, ML models trained on lots of data can be very useful. You only need a few lucky guesses, that you can verify downstream, to end up with some useful result. In short, there is no magic to it. It's a useful tool that can be regarded as both a search algorithm and an optimization algorithm.
I find it interesting that fluid dynamics models struggle more than weather. Intuitively, my view of weather is that it looks a lot like fluid dynamics and seems quite chaotic. Is this wrong or is it just the vast amount of recorded weather data which helps?
edit: I realised just after posting that you only mentioned climate models. It was another poster who said there were successes using ML for meteorology.
I think an important question to ask is whether your scientific task is primarily one of interpolation, or one of extrapolation. LLMs appear to be excellent interpolators. They are bad at extrapolation.
Climate models aren't LLMs.
They're also not AI.
It remains to be seen exactly how much a climate model can be improved by AI. They're already based on woefully sparse data points.
Why are they not AI?
Check out Fourier Neural Operators, they claim to have a pretty solid solver for fluid flow equations (Navier Stokes etc).
I am already acquainted with them but to be honest, I am no longer in the field so I am not able to comment on latest developments. However, as of two years ago, the consistent result was that you could get models that reproduce really good physics for problems in the same physical regimes as the training data, but such models had poor generalizability, so depending on the use case, they weren't of much use. The only exception I know is FourCastNet, which is a weather model FNO from NVIDIA.
Yes. ML has advanced many fields related to modelling - meteorology, climate, molecular. Classification models have done much for genomics, particle physics, and other fields where experiments produce inhumane amounts of data.
DeepVariant, Enformer, ParticleNet, DeepTau, etc. are some well-known individual models that are advanced branches of science. And there are the very famous ones, like AlphaFold (Nobel in Chemistry 2024).
We need to think of AI not as a product (chats, agents, etc.), but as neural nets (AlexNet). Unfortunately, large companies are "chat-washing" these tremendously useful technologies.
ML was used to sharpen the recent image of a black hole: https://physics.aps.org/articles/v16/63
ML is more of a bag of techniques that can be applied to many things than a pure domain. Of course you can study the properties of neural networks for their own sake but it’s more common as a means to an end.
Surely they mean LLMs
Bit short on details other than "Let's see what LLMs can predict when we train them on various scientific data sets."
Certainly a good thing to try, but the article feels like a PR piece more than anything else, as it's not answering anything, just giving a short overview of a few things they're trying with no data on those things whatsoever.
It does fit in with the "Throw LLM spaghetti at a wall and see what sticks" trend these days though.
Should be possible to backtest by training LLMs on historic datasets and then probing them to see if they can re-discover things that were discovered after their training data cut-off. What sort of prompts could push them to make a breakthrough.
It’d be tricky to avoid inadvertently leaking in the prompt since many discoveries seem obvious in retrospect.
I think that’s an opportunity, not a problem. If prompt + hint generates a verifiable solution then you can build systems that propose hints, either randomly or by exploring a search space, and keep trying combinations until you hit on something that works.
exactly. hindsight bias makes it really hard to separate genuine inference from subtle prompt leakage. even framing the question can accidentally steer it toward the right answer. would be interesting to try with completely synthetic problems first just to test the method.
Maybe you could do it with math research?
First, give it the abstract for a fresh paper that it couldn’t have been trained on, then see if it can come up with the same proofs to see if it can replicate the logic knowing the conclusion.
Second, you could give it all the papers cited in the intro and ask a series of leading questions like “based on this work, what new results can you derive”?
AlphaProof is among the most relevant methods here. And because it trains by self-play, instead of historical human data - it has a much better chances of being able to solve novel problems, or come up with solutions that humans have not. It did pretty good at the 2024 Olympiad. Will be interesting to see how 2025 goes.
Honestly, that's still far too much help in lots of cases.
Finding a set of papers, whose results can be combined in a reasonable amount of time to make a new interesting result is itself a hard problem. This is often a thing Professors do for PhD students -- give them a general area to research and some papers to start reading.
It's still a contribution, but so much easier than just asking "Hey, choose a set of papers from which you can derive new interesting results"
I think the challenge here is assembling enough verifiably clean data to train a foundational model.
I thought Alpha Fold advanced biology enough to get Jumper and Hassabis the Nobel Prize in Medicine.
.. it was Chemistry 2024
I feel like there are a few ways in which science could be advanced by AI models.
1. We have the sense that "science progresses one funeral at a time." An AI model could be used to recognize situations where a single viewpoint is getting a disproportionate amount of attention in the literature, and warn journals+funding agencies about any disconnects between attention and quality of work.
2. We have the sense that "It Ain’t What You Don’t Know That Gets You Into Trouble. It’s What You Know for Sure That Just Ain’t So" An AI model could identify the most high-profile results in the literature that have contradicting evidence and call for further, decisive study.
3. Interdisciplinary translation. There are many many cases of different branches of science re-discovering each other's work. I believe I read an article a little bit ago about an academic in a somewhat softer science publishing a paper proudly claiming the discovery of linear regression. Obviously not all cases are so egregious, but an AI could advance a discipline just by pointing out areas where that discipline is using outdated/inferior methods compared to the state of the art.
I think our creativity has not yet been duplicated in AI, so for maximum results, we need to pair AI with a human expert or a panel of human experts and innovate by committee. AI brings to the table vast memory, instant recall and most importantly, tired-less pursuit and the human element can provide creative guidance and prompt. The trick is in curating the BOK(body of knowledge) used to train GENERATIVE AI. I wonder what a curricula designed specifically for AI would look like?
Did… did this really long bio answer the question? I’m not sure what this is trying to do tbh, but it was interesting to see how these labs have been trying to incorporate AI.
AlphaFold already did. Or do we only count AI if it’s an LLM now?
I'm all for a renarrowing of the use of AI to no longer just cover any machine learning algorithm to be honest. It was a really annoying feature of the 2010s that any kind of machine learning application got the hype word du jour slapped on it.
The problem is it is just ML all the way down… I mean even AlphaFold used transformer models, I thought.
And going even further, curve fitting.
I would be interested in machine learning for scientific research. Something more “physical” than optimizing software.
I checked some of the nuclear fusion startups and didn’t see anything.
I got a 404 loading the page, is it blocked outside the US?