AI has returned chipmaking to the heart of computer technology
And the technological challenges are bigger than the political ones!
Sep 16th 2024
Acentury ago, 391 San Antonio Road in Mountain View, California, was the site of an apricot-packing shed. Today it is just one of the many low-rise office blocks on busy roads that house Silicon Valley’s tech startups and wannabe billionaires. In front of it, though, stand three large and peculiar sculptures, two-legged and three-legged forms that bring to mind water towers. They are giant versions of two diodes and a transistor, components of electronic circuitry. In 1956, 391 San Antonio Road became the home to the Shockley Semiconductor Laboratory, a startup devoted to the idea of making such components entirely out of silicon. It is the birthplace of Silicon Valley.
The firm, founded by William Shockley, a coinventor of the transistor, was a commercial flop. The embrace of silicon was not. In 1957 eight of Mr Shockley’s employees, whom he dubbed the “traitorous eight”, defected to start Fairchild Semiconductor less than two kilometres away. Among them were Gordon Moore and Robert Noyce, future co-founders of Intel, a chipmaking giant, and Eugene Kleiner, co-founder of Kleiner Perkins, a ground-breaking venture-capital firm. Most of the storied tech companies in Silicon Valley can trace their roots, directly or indirectly, back to those early Fairchild employees.
Before semiconductor components were invented, computers were room-size machines that used fragile and finicky vacuum tubes. Semiconductors, solid materials in which the flow of electrical current can be controlled, offered components that were more sturdy, more versatile and smaller. And when such components were made mostly from silicon, it became possible to make a whole raft of them on a single piece of the stuff. Tiny transistors, diodes and the like on silicon “chips” could be wired together into “integrated circuits” designed to store or process data.
In 1965 Moore, while still at Fairchild, noted that the number of transistors that could be put into an integrated circuit at a given cost doubled every year (he later relaxed the doubling time to every two years). His observation, codified as “Moore’s law”, mattered. Chips produced in 1971 had 200 transistors per square millimetre. In 2023 the mi300, a processor built by amd, an American semiconductor firm, crammed 150m transistors into the same area. The smaller the transistors got, the faster they could switch on and off. The mi300’s components are thousands of times faster than their predecessors were 50 years ago.
All major breakthroughs in computing, from personal computers and the internet to smartphones and artificial intelligence (ai), can be traced to transistors getting smaller, faster and more affordable. The transistor’s progress has driven technology’s progress.
For a while, the technological centrality of silicon chips was mirrored by the importance of the businesses that made them. In the 1970s IBM, which made chips, the computers that used them and the software that ran on them, was a giant beyond compare. In the 1980s Microsoft proved that a company selling only software could be even more attractive. But Intel, which made the chips on which Microsoft’s software ran, was a huge force in its own right. Before the dotcom bust of 2000 Intel was the sixth-biggest company in the world by market capitalisation.
After the bust, though, the “Web 2.0” services offered by firms like Google and Meta took centre stage, with the semiconductors on which their platforms were built increasingly commodified. To describe the dynamic underlying the growth of big tech, it was software, not silicon, that Marc Andreessen, a venture capitalist, credited in 2011 with “eating the world”.
The bo-m in ai has changed that; its progress depends on immense computational power. Before 2010 the amount of computing needed to train leading ai systems grew roughly in line with Moore’s law, doubling every 20 months. Since then it has doubled every six months (see chart 1). That means there is ever more demand for ever more powerful chips. Nvidia, an American company which specialises in chips of a sort peculiarly well suited to the needs of the large language models (LLMs) that dominate AI, is now the third-most valuable company in the world. Since late 2023 the msci index of chipmaking firms has outperformed its index of software firms by a wide margin for the first time in over a decade (see chart 2).
As AI makes chipmaking important again, companies with AI ambitions are getting into the game themselves. The driver is not just training, but subsequent use (also called “inference”). Answering queries with LLMs, though not as demanding as training them in the first place, is still a big computational task, and one that needs to be undertaken billions of times a day. Because bespoke circuits can do this more efficiently than the general-purpose ones sold by most semiconductor providers, some companies running llms are choosing to design chips just for this purpose. Apple, Amazon, Microsoft and Meta have all invested in building their own custom ai chips; there are more processors designed by Google and used in data centres than by any other company but Nvidia and Intel. Seven of the ten most valuable firms in the world are now in the chipmaking business.
The sophistication of a chip depends mostly on how small its features are; currently the cutting edge is defined as having “process node” measurements of less than seven-billionths of a metre (seven nanometres, or 7nm—see box on later page for a pinch of salt with which to take this). That is where the ai action is centred. But over 90% of semiconductor manufacturing capacity works with process nodes of 7nm or more. These chips are less technologically challenging, but more widespread, found in everything from televisions and refrigerators to cars and machine tools.
In 2021, at the height of the covid-19 pandemic, an acute shortage of such chips disrupted production across various industries, including electronics and cars. The industry’s pursuit of efficiency had seen it become globally distributed, with different regions specialising in different bits of the chain: chip design in America; chipmaking gear in Europe and Japan; the fabs where that gear is used in Taiwan and South Korea; the packaging of the chips and their assembly into devices in China and Malaysia. When the pandemic disrupted these supply chains, governments took note.
In August 2022 the American government dangled a $50bn package of subsidies and tax credits to lure chip manufacturing back to America. Other regions have followed suit, with the eu, Japan and South Korea promising almost $94bn in handouts. Things have been made more complicated by America’s attempts to cut off China’s access to cutting-edge chips and the tools with which they are made by means of export bans. China has responded to those bans by restricting exports of two materials vital for chipmaking.
But the chipmakers’ biggest worries are not industrial policy or national rivalries. They are technological. For five decades, shrinking transistors boosted performance without increasing energy consumption. Now, as chips get denser and ai models bigger, energy use is soaring. To maintain exponential gains in performance, chipmakers need new ideas. Some, like tighter integration between hardware and software, are incremental. Others are radical: rethinking silicon or ditching digital processing for other techniques. This Technology Quarterly will show how such advances can keep the exponential engine humming.
Source:
https://www.economist.com/technology-quarterly/2024/09/16/ai-has-returned-chipmaking-to-the-heart-of-computer-technology
No Paywall:
https://archive.is/j4wBL