So there I was, lost in my musings about the future of computational chemistry. In a tale of boundless imagination, I conjured up the concept of chemGPT—a remarkable large language model for chemistry that could engage in conversation with you while conducting atomistic simulations. Perhaps it was more a flight of fancy than a concrete prediction, but little did I know that my vision wasn’t far-fetched after all. In fact, the advent of chemGPT might already be within our grasp. Today, we venture into the heartland of large language models to discover how they may be shaping the evolution of chemistry. Buckle up, my friends, and let us meet our chemGPT’s.
Large Language models for chemistry?
Whether you are computational chemist, an experimental researcher, or someone who wouldn’t know a pipette from a pineapple, chances are you’ve heard whispers of the mystical beings known as large language models. Oh, but of course you have! You certainly encountered, used, loved, or perhaps hated, chatGPT.
But let’s not get caught up in our personal opinions here. No matter what our take on this is, we must admit that we’ve come a long way with machine learning.
We’re talking about language models that can correctly answer questions, summarize texts, convert files, and do all sorts of phenomenal stuff. You feed one of these models a PDF filled with instructions, and you can ask it technical questions without reading all of it. It can even cite its sources if you give it the right prompts. Plus, these models can write emails, sum up meetings, extract action items from transcripts, and query databases—You just need to ask nicely.
ChatGPT and its cohorts are like the rock stars of natural language processing, using cutting-edge machine learning to generate text that’s so darn good, it’s hard to tell it apart from human writing.
Hello there, Mr. Turing!
From a scientific standpoint, the implications are nothing short of revolutionary. These language models can whip up abstracts for scientific articles with a flick of their digital wrists. They can craft lines of code tailored to specific programming tasks like a virtuoso on a twelve-string. And that’s not all—they can even take on challenges they were never explicitly trained for, like some kind of machine-learning sorcerer.
It’s as if they possess an innate adaptability, an insatiable hunger for tackling fresh obstacles. A chilling thought, if you think about it.
Now, if these large language models can handle tasks they weren’t initially designed for, could they also hold the answers to the scientific questions that have plagued us for centuries?
GPT-4 can accomplish complex tasks in chemistry purely from English instructions, which may transform the future of chemistry
Andrew D. White (Nature Reviews 2023)
Take chemistry, for example. Just imagine being able to ask these language models questions like, “If I swap the metal in my metal-organic framework, will it be moisture-stable?” or “What’s the free energy landscape of that DNA transition?” or again “What is the role of hydrogen bonding in that biological process?”
These challenges can be tackled through atomistic simulations (check this out), but answering them properly requires expertise and a lot of work.
Effortlessly, language models present answers to our burning questions in the blink of an eye, leaving us mortals to wonder.
But are these responses to be trusted? Can these language models correctly kick ass in our beloved chemistry?
More on From Atoms To Words:
▸ Let’s Fight Climate Change With The Computational Design of Metal-Organic Frameworks (MOF’s)
▸ Multiscale Simulations of DNA: From Quantum Effects To Mesoscopic Processes
▸ Water’s Hydrogen Bonds: What Makes Them Vital for Life As We Know It?
The beginning of a new era
Dr. White, not the meth-cooking kind of chemist, but one who’s been tinkering with language models for a while, has shared some strong insights on Nature Reviews. He paints a picture of a future where these language models become the driving force behind chemical innovation, like a metanetwork that connects all tools and communicates with users in simple, everyday language.
It’s not just some pipe dream. These language models are already making waves in reaction synthesis planning. And they’re starting to unravel molecular properties, pushing the boundaries of what we can understand.
For instance, check out this cool experiment by Hocky. They threw a language model a bone and asked it to compute the dissociation curve of H2. And guess what? The language model not only spit out the right code to run the quantum chemistry simulation, but it also plotted the curve. When confronted, the language model switched from the initially chosen poor method (Hartree-Fock) and basis set (STO-3G) to a more suitable basis set and a super quantum accurate method.
Who cares about hydrogen dissociation?, you might say.
Fair point, and yet some do.
But imagine this: you embark on a wild adventure into the territory of next-generation battery design. Your noble mission? To unravel the link between electrode composition and key performance descriptors such as volume change, OCV curve, and enthalpy of formation. You have the option to run atomistic simulations, undoubtedly a respectable path, or seek counsel from your language model. It might give you the answer straight away or write the simulation code for you. Now, wouldn’t that be something!
You could use language models to design better metal-organic frameworks for carbon capturing and CO2 conversion, to run a multiscale simulation approach of potential anticancer agents interacting with DNA, or to guide the discovery of alloy materials with enhanced properties.
The potential is endless. The sky’s the limit.
And yet, hard questions persist: Will the language model provide meaningful results? Are we there yet?
Sadly, not today. But in the future, tapping into the power of language models could save us countless hours of manual labor and make computational chemistry expertise a bit less crucial.
Don’t get me wrong, experts will always have their place. Just like language models won’t replace writers or musicians, they won’t replace computational chemists.
The goal isn’t to shove computational chemists out the door; it’s about supporting their efforts or empowering non-experts, for example, by lowering the barrier into atomistic simulations.
And that, my friend, could pave the way for a silent revolution in the widespread use of computer-aided design in R&D.
Alright, enough with the fluff. It’s time to dive into some intriguing examples. Are you ready to explore three remarkable success stories of large language models for chemistry? Let’s go.
More on From Atoms To Words:
▸ When Will RNA Structure Prediction Get Its AlphaFold Breakthrough?
▸ Predicting The Hydrogen Dissociation Energy: The 100-Year Battle of Quantum Chemistry vs. Experiment
▸ All-Atom Molecular Dynamics of SARS-CoV-2: The Computational Microscope’s View of 305 Million Atoms
3 Success stories of large language models for chemistry
1. The Omniscient Chemist
Kevin Maik Jablonka is a human chemist from EPFL, Switzerland, who fearlessly delved into the realm of finely tuned language models, demonstrating their above-average accuracy in tackling a wide range of scientific topics like material properties, synthesis techniques, and design principles. He went deep into HOMO-LUMO gaps, solubility phenomena, organic photovoltaics, and even had the courage to scrutinize alloys, metal-organic frameworks, and polymers. And guess what? The language model’s predictions went beyond the training data, conjuring up structures with desired HOMO-LUMO gaps that blew everyone’s minds.
Jablonka concluded that the ability of language models to extract meaningful insights from natural language inputs opens up new possibilities for chemists and material scientists. While the precise reasons behind their effectiveness remain a bit fuzzy, their potential to revolutionize the field is nothing short of extraordinary.
2. The interpreter
In the world of drug discovery and medicinal chemistry, deep learning models have been shaking the field of quantitative structure-activity relationship (QSAR) modeling. But there’s always been a pesky problem: these deep learning models are often enigmatic black boxes, lacking that crucial ingredient of providing scientific insights behind their predictions.
Enter the study of Gandhi and White (seriously, these names are epic!).
These brilliant minds unleashed a language model that can penetrate those black boxes, decode their secrets, and come back with a scientific explanation. They concocted an explanation method that combines interpretable chemical fingerprints and molecular descriptors, acting as symbolic representations for molecules themselves. And what did they discover? The influence of molecular substructures on key descriptors unveiled itself, providing practical scientific insights.
3. The coder
We all know language models can do some serious coding. But can they handle quantum chemistry?
Dr. White and his crew decided to put that to the test. They meticulously crafted chemistry problems and let the language models work their magic. The models were judged based on the accuracy of the code they churned out, with automated testing and the scrutinizing eyes of human experts. And let me tell you, these language models showcased their coding prowess across a vast array of chemistry topics, leaving everyone dumbfounded. Astonishingly, they could even address their own missteps, adeptly navigating error messages or rectifying mistakes when guided by the hand of a human user.
In their work, Dr. White and his crew stressed that these language models, though undeniably impressive, are not meant to replace DFT calculations or the computational chemists who run it. Rather, they serve as conduits, facilitating seamless communication between individuals and the tools at their disposal.
Language Models for Chemistry: Big Data? Big Problem!
There’s a growing concern floating around that we may have gone a little too far with this whole AI thing. Even Geoffrey Hinton, a big shot in the AI world, recently stepped down from his position at Google to raise awareness about the potential dangers this technology might bring to society.
Now, amidst these overarching concerns, there’s another frustration bubbling up in various scientific domains, including the world of chemistry. And that frustration is the realization that AI hasn’t quite lived up to the lofty expectations we had for it.
Chemistry applications require computer models to be better than the best human scientist. Only by taking steps to collect and share data will AI be able to meet expectations in chemistry and avoid becoming a case of hype over hope.
Nature Editorial 2023
Here’s the deal: The effectiveness of any AI system relies critically on the quality of the data it’s trained on. So, if we want to fully unleash the power of generative AI tools in chemistry, guess what? Chemists need to step up and contribute to building these mighty training datasets.
We need more data. Experimental data, simulated data, historical records, and hell, even knowledge gleaned from failed experiments. And we need to make sure all this juicy information is easily accessible.
Take, for example, an incredible story of AI success: the AlphaFold protein-structure-prediction tool. This powerhouse was trained on a massive dataset—the motherlode of structural information known as the Protein Data Bank. This beast has been collecting experimentally determined protein structures since freaking 1971 and currently boasts over 180,000 structures. AlphaFold is a prime example of what AI systems can accomplish when armed with a truckload of top-notch data. But why hasn’t this approach worked (yet) with RNA? The answer: a lack of data.
So, how can we get other AI systems to generate or access even larger quantities of well trimmed chemistry data?
There are some notable actions being taken to tackle this issue. We got the University of Cambridge (UK) stepping up with their organic chemistry database, Nature, the scientific journal, banging the drum and urging authors to deposit their data in open repositories, and let’s not forget the recent open reaction database.
The only way AI can live up to the hype and meet the sky-high expectations in the field of chemistry is by actively collecting and sharing data. It’s time to roll up our chemical sleeves and make it happen.
More on From Atoms To Words:
▸ 60 Years in the Making: AlphaFold’s Historical Breakthrough in Protein Structure Prediction
▸ AI in Drug Discovery: Chasing Dreams, Facing Realities
▸ Is Machine Learning Going to Replace Computational Chemists?
A final personal touch
Close your eyes and envision a world where our tools, our data, and our triumphs all dance together in perfect harmony. It’s a fluid, graceful movement guided by the power of natural language.
Just imagine it for a moment. All barriers obliterated. The ultimate democratization of computational chemistry, of chemistry itself, and dare I say, of science as a whole. It’s a world where anyone, regardless of their background or expertise, can engage with the beauty and intricacies of Nature. It’s a world where scientific exploration is open and accessible to all who dare to dream.
Together, humans and machines, we can revolutionize the way we approach science and reshape the very fabric of scientific advancement.
But let’s not forget the potential pitfalls of technology. After all, we wouldn’t want a blond replicant ranting about “attack ships on fire off the shoulder of Orion” while assisting us with our chemistry simulations.
Or would we?
If you enjoyed this dive into large language models and their incredible potential, I’d love to hear your thoughts. Agree, disagree, or have a totally wild theory of your own? Let’s connect! Subscribe to my LinkedIn newsletter and let’s keep the conversation rolling.