Generative artificial intelligence (AI) in the form of natural-language processing technology has taken the world by storm, with organizations large and small rushing to pilot it in a bid to automate tasks and increase production.
Tech giants Google, Microsoft, and Amazon are all offering cloud-based genAI technologies or baking them into their business apps for users, with global spending on AI by companies expected to reach $301 billion by 2026, according to IDC.
But genAI tools consume a lot of computational resources, primarily for training up the large language models (LLMs) that underpin the likes of OpenAI’s ChatGPT and Google’s Bard. As the use of genAI increases, so too does the strain on the hardware used to run those models, which are the information storehouses for natural language processing.
Graphics processing units (GPUs), which are created by connecting together different chips — such as processor and memory chips — into a single package, have become the foundation of AI platforms because they offer the bandwidth needed to train and deploy LLMs. But AI chip manufacturers can’t keep up with demand. As a result, black markets for AI GPUs have emerged in recent months.
Some blame the shortage on companies such as Nvidia, which has cornered the market on GPU production and has a stranglehold on supplies. Before the rise of AI, Nvidia designed and produced high-end processors that helped create sophisticated graphics in video games — the kind of specialized processing that is now highly applicable to machine learning and AI.
AI’s thirst for GPUs
In 2018, OpenAI released an analysis showing since 2012, the amount of computing power used in the largest AI training runs had been increasing exponentially, doubling every 3.4 months (By comparison, Moore’s Law posited that the number of transistors in an integrated circuit doubles every two years).
“Since 2012, this metric has grown by more than 300,000x (a 2-year doubling period would yield only a 7x increase),” OpenAI said in its report. “Improvements in compute have been a key component of AI progress, so as long as this trend continues, it’s worth preparing for the implications of systems far outside today’s capabilities.”
There’s no reason to believe OpenAI’s thesis has changed; in fact, with the introduction of ChatGPT last November, demand soared, according to Jay Shah, a researcher with the Institute of Electrical and Electronics Engineers (IEEE). “We are currently seeing a huge surge in hardware demands — mainly GPUs — from big tech companies to train and test different AI models to improve user experience and add new features to their existing products,” he said.
At times, LLM creators such as OpenAI and Amazon appear to be in a battle to claim who can build the largest model. Some now exceed 1 trillion parameters in size, meaning they require even more processing power to train and run.
“I don’t think making…
2023-09-30 11:24:03
Post from www.computerworld.com rnrn