I’ve been running a Large Language Model (LLM) similar to GPT-4 on my desktop PC since yesterday. When I say that I’m running it on my desktop, I mean that the entire LLM is stored on my PC. All of it.
We have much to discuss.
Researchers at the Stanford Institute for Human-Centered Artificial Intelligence (HAI) released Alpaca: A Strong, Replicable Instruction-Following Model, which they described as follows:
We introduce Alpaca 7B, a model fine-tuned from the LLaMA 7B model on 52K instruction-following demonstrations. On our preliminary evaluation of single-turn instruction following, Alpaca behaves qualitatively similarly to OpenAI’s text-davinci-003, while being surprisingly small and easy/cheap to reproduce (less than $600).
(For reference, “text-davinci-003” is one of the LLMs that OpenAI used to improve ChatGPT.)
One major criticism of all LLMs is that because they are “large,” they require enormous amounts of computational power, which translates into enormous hardware and energy costs. Because they are so resource-intensive, LLMs also inspire fears that only the biggest of Big Tech will be able to commercialize the technology.
To give you a sense of the exponential pace of innovation, here’s an unofficial timeline:
- 01 FEB: Cathy Woods and ARK Investments publish a report that predicts it will take seven years (by 2030) to train GPT-3-level performance for $30 (down from the estimated 2022 cost of $450,000)
- 24 FEB: LLaMA is announced, starts being shared with academic partners
- 02 MAR: Someone posts a link to models 7B, 13B, 30B, and 65B
- 10 MAR: First commit to llama.cpp by Georgi Gerganov
- 11 MAR: llama.cpp now runs the 7B model on a 4 GB Raspberry Pi (via @miolini)
- 12 MAR: npx dalai LLaMA
- 13 MAR: llama.cpp on a Pixel 6 phone
- 13 MAR: Alpaca (from HAI) with a total cost under $600
If you don’t understand the above timeline… On February 1, 2023, Cathy Woods and ARK Investments thought that OpenAI and other Big Tech companies had roughly a seven-year window before the technology became so cheap that anyone could do it. Last week (just six weeks from their publication date), the hard cost to train a model to GPT-3-level performance was less than $600.
We are still a few weeks away from “anyone can do it,” but if you want to play along at home, the easiest way to run LLaMA and Alpaca on your computer is to install Dalai.
Now, for the part that no one wants to hear – the disclaimer on the HAI post:
We emphasize that Alpaca is intended only for academic research and any commercial use is prohibited. There are three factors in this decision: First, Alpaca is based on LLaMA, which has a non-commercial license, so we necessarily inherit this decision. Second, the instruction data is based on OpenAI’s text-davinci-003, whose terms of use prohibit developing models that compete with OpenAI. Finally, we have not designed adequate safety measures, so Alpaca is not ready to be deployed for general use.
Models training models, training models, training models… this is all very meta, but it is the Dalai LLaMA, after all.
This is going to spin out of control in very short order. OpenAI’s APIs are priced by usage; that’s how they make money. Will they restrict my usage? It seems unlikely. If they don’t, I will build models based on their models. If they do, I’ll find other models to train on. It may be time to short Big Tech AI and instead go long on synthetic data (which seems like it’s going to be the new, new thing).
Author’s note: This is not a sponsored post. I am the author of this article and it expresses my own opinions. I am not, nor is my company, receiving compensation for it.