You could have invested a lot of money in an AI chip startup a few years ago, or you could have put that money into Nvidia stock and made a lot more money.
Take Graphcore, for example: In December 2018, the company raised $200 million from investors including Microsoft, Samsung, and leading European VC firm Atomico.
Graphcore is designing chips to replace Nvidia GPUs for AI workloads, and the startup has also created a complete software stack called Poplar to take on Nvidia’s dominant CUDA platform.
So far, the plan hasn’t worked out: Graphcore warned of “going concern uncertainties” in a regulatory filing last year.
It’s fair to say it wasn’t a good investment. What would have happened if these investors had put $200 million into Nvidia in late 2018?
Since then, Nvidia’s stock price has risen by about 3,000% — turning $200 million into $6.2 billion — a $6 billion gain in five and a half years.
“It’s a very foolish investment.”
“Betting on Nvidia is considered a very foolish investment,” says one veteran venture capitalist who asked to remain anonymous.
So what does Silicon Valley do best? Often, Silicon Valley thrives on true technologists continuing to work on things they believe in, even if everyone else thinks they’re stupid.
Now, a string of startups are poised to challenge Nvidia at a time when the chip giant is at its most powerful.
“It would be insane for us to go to war with them,” said Thomas Sommers, CEO of Positron AI, a semiconductor startup that began operating in secret a few months ago.
NVIDIA Mountain Climbing
When Thomas Sommers was a teenager, he sat at a desk surrounded by computer monitors. MediaNews Group/Boston Herald via Getty Images
Given the mountain Positron has to climb, Sommers and his colleagues are taking a relatively capital-efficient approach initially, but they have bigger, long-term plans.
My anonymous venture capitalist said the Positron team was “incredible.” Somers was a tech genius who had worked in an MIT lab since he was 13.
For now, Positron is focused solely on Transformer models, the architecture that underpins OpenAI’s GPT model, which is the basis for ChatGPT.
Most of the recent AI breakthroughs have come from these models, and Positron has designed a server called Atlas with eight specialized AI chips. These components have much more memory than Nvidia GPUs, making them a good configuration for Transformer models, Sohmers told BI.
Positron is also getting started on inference, which is the process of creating an AI model and then running, monitoring and evaluating it as it receives requests and serves up output. Training an AI model is a huge undertaking and expensive, but Sohmers and his team expect the inference market to grow over time.
What about the software layer?
For now, Positron supports the library of Transformer models available on Hugging Face, a popular hub for open-source AI models. Sohmers says there are currently more than 100,000 of these models, which is a healthy market for Positron at the moment.
Hardware-wise, the startup is using the Swiss Army knife of the semiconductor industry: field-programmable gate arrays (FPGAs): Even after these chips are manufactured and installed, they can be programmed for other uses.
That will allow Positron to get its first products out the door sooner than other AI-chip startups, Somers said. Just one year after Positron was founded, the company is shipping its first-generation Atlas Server to early paying customers this month.
“The crazy model is that AI chip startups need to raise tens or hundreds of millions of dollars before they can actually test their product for market fit,” he added, “and then by the time they ship their first or second generation chips, they realize they’ve actually built the wrong thing.”
Positron has raised about $12 million and has 20 employees, and Somers said it has already spent about half of the money it has raised so far.
From TPU to Groq
About half of Positron’s staff comes from Groq, a longer-established AI chip startup.
Groq has raised $367 million to date and is seeking further funding. The company was founded in 2016 by former Google engineers, some of whom helped design the TPU, Google’s answer to Nvidia GPUs.
The company also focuses on the inference market and provides software support through GroqWare and the Groq compiler (a compiler translates code written in one programming language into another).
Chrome to Rivos
When taking on the dominant AI development platform, it helps to have experts with previous experience building digital platforms.
Rivos is run by ex-Googlers who helped develop the internet giant’s Chrome operating system and accompanying hardware such as Chromebook laptops.
Rivos CEO Puneet Kumar was a senior director at Google where he worked for over 10 years, Rivos Chief Strategy Officer Mark Hayter spent almost 11 years at Google working on Chrome hardware, and Rivos software lead Andrew de los Reyes spent 14 years at Google.
Rivos combines a CPU, the main chip that performs most standard computing tasks, with a GPU optimized for large language models, and uses the open-source RISC-V license instead of the Arm license.
The startup is also looking at CUDA, saying it has a software stack that allows applications and AI models to be recompiled without having to be redesigned.
The company raised $250 million in April from investors including Matrix Capital, Intel and MediaTek.
The funding will go towards tape-out of Rivos’ first chips, which is the final stage of the design process before components are sent for manufacturing. Rivos is also expanding its manufacturing operations and investing in its platform hardware, software engineering and support capabilities.
From Harvard to Etching
Etched was co-founded by two Harvard University mathematicians, Gavin Uberti and Chris Zhu.
Uberti specializes in AI compilers, which should help you outperform Nvidia’s CUDA software system.
Etched is designing GPUs specifically for the Transformer architecture.
AI is falling
Rain AI was founded in 2017 and is backed by OpenAI’s Sam Altman. In early 2022, the startup raised $25 million in new funding.
The company is also working on AI accelerator chips with scalable designs for model training and inference.
A look at Rain AI’s website this week revealed that the startup also licenses intellectual property for its chip designs and software stack.
“Hardware will be available shortly,” it says.
A lot of money. A big tip.
Cerebras is another AI chip startup that has raised huge amounts of funding: Founded in 2016, it had raised $720 million by late 2021.
That huge amount of funding has gone into developing bigger chips, literally: In March, the company unveiled a new AI chip for model training that contains 4 trillion transistors and is 56 times larger than Nvidia’s H100 GPU.
The startup also offers a software platform called the Cerebras Software Framework, which supports PyTorch, a popular AI library, and cutting-edge techniques such as multi-modal models and mixing of experts.
Tigris
OpenAI’s Altman may also be working on his own AI chip startup.
Late last year, Bloomberg News reported that Altman was seeking to raise billions of dollars for a new chip venture codenamed “Tigris.”
The idea sounds familiar: Build semiconductors to compete with Nvidia GPUs.
Masa’s Izanagi
Who can match Altman in tech audacity? SoftBank’s Masayoshi Son.
Son invested billions in the now-defunct WeWork. He was also an early investor in Alibaba and made a huge fortune there. SoftBank also owns Arm, a leading chip-licensing company. And Son tried unsuccessfully to buy Nvidia a few years ago.
So it’s no surprise that Masa is trying to raise capital for his AI chip startup.
Earlier this year, Bloomberg reported that Son was seeking as much as $100 billion in investments in companies that provide semiconductors for AI.
Code name: Izanagi.
