Replacing every Google search with an LLM interaction uses as much electricity as Ireland, Alex de Vries writes in a commentary published in the journal Joule on October 10, 2023. The PhD candidate at Vrije Universiteit Amsterdam is raising concerns about the environmental impact of artificial intelligence.
Large models like ChatGPT, Bard, and Claude consume a lot of energy. Take, for example, ChatGPT. The chatbot is built on a model called GPT-3, which used an estimated 1,287 MWh in electricity during its training phase. That’s the equivalent of driving a Tesla Model 3 around the Earth’s equator 21,500 times, according to the AI.
The chatbot first suggested using households to illustrate the scale of energy consumption. It was a good idea, but its answer was clearly wrong. It doesn’t make sense that 1,287 MWh powers either 1,468 American homes for a year or 122 for a month. The machine politely apologised when I pointed this out.
That’s nice, but I no longer trusted the machine and had to verify its second suggestion – with a Google search. Right now, LLMs are probably increasing rather than replacing traditional search engine traffic. But I digress.
It’s well-known that training an LLM uses a lot of energy, yet that’s only the first step in the process. The second step is inference, a phase few of us have heard of, but many participated in.
For ChatGPT, inference began when it was launched. It continued to learn while interacting with the public and creating live responses. The AI’s estimated energy consumption during this phase was 564 MWh daily. That’s close to half the electricity consumed in training (44%) – but used every day.
De Vries’ concerns stem from the popularity of LLMs. In an interview he elaborates, “Looking at the growing demand for AI service, it’s very likely that energy consumption related to AI will significantly increase in the coming years.”
ChatGPT exploded as soon as it launched, registering an incredible 100 million users in just two months and igniting a chain reaction of artificial intelligence products. It would be convenient if we could skip the inference phase for new AIs but, unfortunately, we can’t because, without it, the machine would not have the ability to learn.
Inference made up a whopping 60% of Google’s AI-related energy bill from 2019 to 2021. But we’re making progress. Hugging Face, a relative newcomer founded in 2016, developed the open-source alternative BLOOM using significantly less energy in the inference phase relative to the training phase.
Let’s take a look at the energy consumption for a single user request. Comparing the different methods, the graph looks as follows.
The last two bars show estimations for AI-powered Google Search by two independent research firms, New Street Research and SemiAnalysis. It’s very costly at 20 to 30 times the usage of regular Google Search. That’s not an immediate problem, however, because NVIDIA can’t supply the hardware required.
hardware constraints
Google would need 512,821 of NVIDIA’s AI servers to make every search an LLM interaction. That’s more than five times the company’s production for 2023, when it’s expected to deliver around 100,000 servers. The gap is enormous. What’s more, NVIDIA has around 95% market share, so there is no alternative supplier.
Chips are an issue, too. NVIDIA’s chip supplier, TSCM, is struggling to expand its chip-on-wafer-on-substrate (CoWoS) packaging technology, which is essential for the chips NVIDIA needs. TSCM is investing in a new plant, but it will only begin to produce volumes in 2027. By then, NVIDIA could have demand for 1.5 million of its AI servers.
Hardware will remain a bottleneck for several more years. Still, without hardware constraints, we’d encounter problems further upstream. Building permissions for data centres take time, and construction does, too. Not to mention, the energy grid needs to expand to deliver the electricity required to run and cool them.
innovation
But these constraints will drive innovation. We will find more efficient models and ways to operationalise them. A breakthrough in quantum computing could change everything, both for the supply and demand for AI.
De Vries points out, “The result of making these tools more efficient and accessible can be that we just allow more applications of it and more people to use it.”
He’s referring to Jevons’ Paradox, which occurs when an increase in efficiency causes costs to fall and demand to increase to the point where we use a resource more than we would have without the improvement.
The paradox was formulated in 1865 and has been observed many times since. LED lighting is a nice example: running LEDs is so cheap that we’ve covered the planet with them and now use more electricity for lights than ever before. If that’s hard to imagine, just take a look at Sphere in Las Vegas.
How amazing is that? But we wouldn’t have built it if the technology hadn’t graduated to LEDs. AI might follow a similar path. Nevertheless, without game-changing innovation to reduce energy use and ensure sustainable supply, we must think twice.
“The potential growth highlights that we need to be very mindful about what we use AI for. It’s energy-intensive, so we don’t want to put it in all kinds of things where we don’t actually need it,” de Vries offers as a parting thought.
Source: Joule via EurekAlert!
For more about new developments in tech and AI, click here.