Galactica: the Language Model that Wrote Its Own Scientific Paper
When you want to know something, where do you turn? If you’re like most people, you’ll probably reach for your phone and type your question into Google,…
When you want to know something, where do you turn? If you’re like most people, you’ll probably reach for your phone and type your question into Google, which will direct you to some human-written articles on related topics.
But when you pose your question to Galactica, the new language model from Meta AI, something radically different happens. Rather than simply returning a list of relevant articles, Galactica will write an original scientific article that answers your question directly, explains the answer in detail, and provides citations for the papers that it used in its answer.
Image: Aron Brand x DALL·E 2
How is that even possible? Well, Galactica has been trained on the sum of humanity’s scientific knowledge, consisting of a large, curated scientific corpus of research papers, reference material, knowledge bases and many other sources. It compresses all this knowledge into a 120-billion parameters model that can fit on a single NVIDIA A100 GPU. I’m kidding you not, the entire knowledge of humanity is condensed in such a form that it fits into 80GB of memory !
And it’s not just a regurgitation machine; when Galactica reaches a calculation that it can’t solve reliably, it writes a program that is then executed on a classical computer. This allows it to solve equations and answer computational questions that are beyond the scope of current AI technology.
In an incredible and somewhat scary twist, the researchers openly admit that Galactica was used in the writing of the paper that announced its existence to the world.
Since Galactica was released as open source, anyone can access and run the model. This is incredible news for the scientific community, and you can be sure that dozens of startup companies will spring up to offer Galactica-based services to the public.
It’s not hard to imagine a future in which Galactica and its “generative search engine” successors are the go-to source for all of our information needs. Will AI replace search engines as the way we find information? Some challenges need to be overcome first, but I believe it’s only a matter of time.
The Shift, in your inbox
Occasional essays on AI, cloud, and the systems shaping what comes next. Subscribe on LinkedIn — no spam, unfollow anytime.
Keep reading
When ChatGPT Beat an 80-Year Erdős Record
A deceptively simple question in combinatorial geometry stumped mathematicians for decades. A general-purpose AI found a new arrangement - and a proof that Erdős himself believed was impossible.
From “Westworld” to Real World: Generative Agents Bringing Fiction to Life
New Stanford research offers a convincing simulation of human behavior in society
Every Reachable System Is Now Worth Attacking
In 1883 a Dutch linguist and cryptographer named Auguste Kerckhoffs laid down what became the closest thing computer security has to scripture: a system should remain secure even if everything about it, save the key, is public knowledge.
