From Celestial AI to Taalas, these startups are looking to challenge Nvidia’s AI computing dominance or find other areas where they could disrupt the semiconductor industry.
About two years ago, as the world woke up to the revolutionary capabilities of AI generation and the powerful chips that make it possible, the semiconductor industry saw renewed interest. The biggest beneficiary of this interest has been Nvidia, but various startups are either challenging the AI ​​chip giant or trying to find other areas where disruption is likely to occur.
[Related: Analysis: As Nvidia Takes AI Victory Lap, AMD Doubles The Trouble For Intel]
While the risks remain high for semiconductor startups — whose costs are typically much higher than those for early-stage software companies — they could benefit from expectations that the industry will grow as much as 20% this year, thanks in part to strong demand for AI chips.
That leaves room for AI chip startups such as Cerebras Systems, Hailo and Kneron to capture some of the spending growth that research firm IDC predicted in December. Other semiconductor startups looking to revolutionize how chips are designed for AI computing include Celestial AI, Eliyan, Rivos and Tenstorrent.
Below are CRN’s 10 hottest semiconductor startups for 2024 so far, which in addition to the startups mentioned above also include MetisX, SiMa.ai and Taalas.
Celestial AI
chief executive officer: David Lazovsky, Founder and CEO
Celestial AI says its Photonic Fabric optical interconnect technology overcomes latency and bandwidth bottlenecks, paving the way for advancements in AI computing.
The Santa Clara, California-based startup announced in March that it had raised a “highly oversubscribed” $175 million in a Series C funding round led by U.S. Innovative Technology Fund and backed by several investors including AMD’s venture arm, Samsung and Volkswagen Group holding company Porsche SE.
That same month, a silicon photonics startup said hyperscalers and semiconductor companies, the world’s largest consumers of datacenter infrastructure, “are currently designing photonic fabric optical chiplets as early adopters of the technology.” Celestial AI said the integration of optical chiplets within multi-chip packages is increasingly becoming the standard for high-performance processors, potentially enabling up to 25 times greater out-of-package bandwidth compared to “other state-of-the-art technologies.”
Celebras Systems
chief executive officer: Andrew Feldman, Co-Founder and CEO
Cerebras Systems is challenging Nvidia’s AI computing dominance with its Wafer Scale Engine chips, which it says will deliver superior performance per watt and “unprecedented scalability.”
The Sunnyvale, California-based startup unveiled its third-generation chip, the Wafer Scale Engine 3, in March, which the company said offers “twice the performance” of the previous generation at the same power and price. Built with 4 trillion transistors and built on TSMC’s 5-nanometer process, the WSE-3 packs 900,000 AI cores, 44GB of on-chip SRAM, and delivers 125 petaflops of 16-bit floating-point performance.
Other milestones for the company this year include a multi-year strategic partnership with healthcare giant Mayo Clinic to develop multi-modal large-scale language models to improve patient outcomes and diagnostics; a multi-year partnership with AI startup Aleph Alpha to build safe, independent AI solutions; and the breaking ground on the Condor Galaxy 3 supercomputer for Abu Dhabi-based technology holding group G42.
Elion
chief executive officer: Ramin Farjadrad, Co-Founder and CEO
Eliyan hopes to use its NuLink PHY interconnect technology to break down die-to-die bandwidth barriers and help chip designers build more powerful chiplet-based processors.
The Santa Clara, California-based startup announced in March that it had closed a $60 million funding round co-led by Samsung Catalyst and Tiger Global Management, with backing from other investors including Intel’s venture arm and SK Hynix.
Earlier this year, Eliyan announced it had taped out its “highest performing” solution at the physical layer connecting multiple dies in a single-chip architecture, delivering up to 64Gbps per link using TSMC’s 3-nanometer manufacturing process.
Hi-Ro
chief executive officer: Co-founder and CEO, Eau Danone
Hailo is challenging Nvidia by accelerating generative AI workloads at the edge with chips that lead the way in optimizing performance for cost and power.
In April, the Tel Aviv, Israel-based startup announced it had raised $120 million from investors as an extension of its Series C funding round, in addition to the release of its new Hailo-10 acceleration, which enables “maximum GenAI performance at minimal power” on devices such as PCs and automotive infotainment systems.
For example, the company says the Hailo-10 can run the 7 billion parameter Llama 2 model at up to 10 tokens per second while consuming just 5 watts of power. The chip can also produce an image of the Stable Diffusion 2.1 model in under 5 seconds in the same power envelope.
Kuneron
chief executive officer: Albert Liu, Founder and CEO
Kneron aims to reduce Nvidia’s influence when it comes to generative AI with AI chips designed to cut companies’ server costs and lower PC prices and energy consumption.
The San Diego, California-based startup announced the launch of its second-generation “edge GPT” server, the KNEO 330, in June. The company said the server can reduce AI costs for small and medium-sized businesses by 30 to 40 percent. With 48 tera-operations per second (TOPS) and up to eight simultaneous connections, it supports large language models and search-enhanced generation accuracy on par with cloud solutions.
Kneron, which has raised $190 million in funding from investors including Qualcomm and Foxconn, has also released its third-generation neural processing unit (NPU), the KL830, designed to enable low-cost AI PCs and AI-enabled IoT applications.
Metis X
chief executive officer: Jim Kim, Co-Founder and CEO
MetisX aims to make data centers “smarter, faster and more cost-effective” by developing intelligent memory systems based on its Compute Express Link (CXL) technology.
The Seoul, South Korea-based startup announced in May that it had raised a $44 million Series A funding round from a range of investors and said it planned to set up a U.S. base next year and introduce chips aimed at hyperscale customers.
The company has already completed prototypes for large-scale data processing use cases such as vector databases, big data analysis, and DNA analysis, and reported that it has discovered prototypes that deliver twice the performance of conventional server CPUs.
Libos
chief executive officer: Puneet Kumar, Co-Founder and CEO
Rivos wants to revolutionize the data center market with chips that combine high-performance RISC-V CPUs and data-parallel accelerators for data analytics and generative AI workloads.
The Santa Clara, California-based startup, founded by former Google, Apple and Intel engineers, announced in April that it had raised more than $250 million in a Series A-3 funding round from investors, including Intel and Dell Technologies’ venture arm.
The company announced news of the funding after reaching a settlement in a lawsuit with Apple in February. It said it would use the money to tape out its first silicon product and expand its team. Apple had accused the startup of stealing trade secrets by hiring dozens of engineers from the tech giant, but Livos had countersued with unfair competition.
Shima
chief executive officer: Krishna Rangasaii, Founder and CEO
SiMa.ai hopes to supplant Nvidia in generative AI workloads at the edge with powerful, efficient chips that can handle a range of modalities on one “software-centric” platform.
The San Jose, California-based startup announced in April that it had raised $70 million from investors including Dell Technologies’ venture arm and Cadence Design Systems Inc. Executive Chairman Lip Vu Tan.
The company said it will use the funds to continue meeting customer demand for its first-generation machine learning system-on-chip (MLSoC) focused on computer vision, as well as to accelerate development of its second-generation MLSoC supporting multi-modal generative AI workloads, including voice, audio, text and images.
Ten Torrent
chief executive officer: Jim Keller CEO
Tenstorrent aims to blaze a new trail in chip design for AI computing, with a business model that combines selling specialized processors and licensing its chip technology for other companies to use.
The Toronto, Ontario-based startup announced in February that it had signed a “multi-tiered partnership agreement” with Japan’s Advanced Semiconductor Technology Center. The center will leverage Tenstorrent’s RISC-V and chiplet technology for its 2-nanometer edge AI accelerators. The startup will also serve as a co-design partner for the chips.
TenStorrent, which raised $100 million in a funding round last year led by Hyundai Motor Group and Samsung Catalyst Fund, is set to raise at least $300 million in a new round led by Samsung, valuing the company at $2 billion, The Information reported in June. LG Electronics, another major South Korean company, is also reportedly in talks for the round.
Talas
chief executive officer: Ljubisa Bajic, Founder and CEO
Taalas is looking to disrupt Nvidia’s general-purpose GPU strategy by designing accelerator chips that directly implement entire AI models, which it says could reduce costs by up to 1,000 times.
The Toronto, Ontario-based startup, led by Tenstorrent founder Llubisa Bajic, announced a $50 million funding round in March and revealed plans to create an automated workflow for incorporating any kind of deep learning model onto chips.
This design approach allowed Taalas to design a chip that can house large AI models in their entirety “without the need for external memory,” the startup says. As a result, the chip design is predicted to deliver higher performance than small-scale GPU data centers, paving the way for a 1,000x or greater reduction in AI computing costs.
The company said it expects to tape out its first large language model chip in the third quarter and have it available to customers in the first quarter of next year.
