The chief scientist at OpenAI, whose short-lived but spectacular November revolt nearly toppled CEO Sam Altman, is starting his own AI company.
Ilya Sutskever revealed on Wednesday that he is teaming up with his OpenAI colleague Daniel Levy and former Apple AI executive Daniel Gross to found Safe Superintelligence, a name chosen to reflect its purpose.
“SSI is our mission, our name and our entire product roadmap – because it is our sole focus,” the trio wrote in a statement posted on the US startup’s no-frills website, adding that building a secure superintelligence is “the most important technological problem of our time.”
Artificial superintelligence (ASI) is considered the ultimate breakthrough in AI, as experts predict that once machines reach a level of human-like general intelligence, known as AGI, their development will be unstoppable.
Starting a new company: https://t.co/BG3K3SI3A1
— Ilya Sutskever (@ilyasut) June 19, 2024
Prominent figures in the field, such as computer scientist Geoffrey Hinton, believe ASI is an existential danger to humanity, and building safeguards that are in our best interest as a species was one of Sutskever’s biggest missions at OpenAI.
His high-profile departure in May came nearly six months after he and independent directors Helen Toner, Tasha McCauley and Adam D’Angelo removed Altman as CEO against the wishes of Chairman Greg Brockman, who resigned immediately.
Sutskever came to regret her role in Altman’s temporary exile.
This meteoric success, which Tonner recently blamed on Altman’s fraudulent activities, threatened to tear the company apart. Sutskever quickly expressed regret and reversed course, demanding that Altman be reinstated to prevent OpenAI’s collapse.
I deeply regret participating in the Board’s actions. It was never my intention to cause harm to OpenAI. I love everything we have built together and will do everything in my power to reunify the company.
— Ilya Sutskever (@ilyasut) November 20, 2023
Toner and McCauley subsequently resigned from the nonprofit’s board, while Sutskever appeared to disappear from public view until announcing her resignation last month.
In his resignation announcement, he suggested he would be working on a project that was “very personally meaningful to me” and promised to release details at a later, unspecified date.
Nonetheless, his resignation set off events that quickly revealed serious governance problems that confirmed the board’s initial suspicions.
First, Sutskever’s co-leader Jan Rijke resigned, accusing the company of breaking a promise to provide 20% of its computing resources to the AI safety team, and then it was revealed that OpenAI employees had been placed under strict gag orders forbidding them from criticizing the company after they left, and were penalized with the loss of vested interests.
Finally, there’s actress Scarlett Johansson, who played an AI chatbot in the 2013 sci-fi film directed by Spike Jonze. she—She then sued OpenAI, alleging that the company had stolen her voice and used it in its latest AI product. OpenAI denied the allegations but promised to respect her wishes and change the voice.
After nearly a decade, I have decided to leave OpenAI. The company’s trajectory has been nothing short of miraculous, and I am confident that under his leadership OpenAI will build safe and beneficial AGI. @Mr, translation:, Miramurati And now, under…
— Ilya Sutskever (@ilyasut) May 14, 2024
These cases suggest that OpenAI has abandoned its original goal of developing AI that benefits all of humanity, and instead pursued commercial success.
“People who were interested in safety, like Ilya Sutskever, wanted to see a lot of resources put into safety, but people who were interested in profits, like Sam Altman, didn’t want that,” Hinton told Bloomberg last week.
A leader in this field since the AI Big Bang era
Sutskever has long been one of the brightest minds in the field of AI, working on artificial neural networks that conceptually mimic the human brain to train computers to learn and abstract from data.
In 2012, he teamed up with Hinton to collaborate on what is generally considered AI’s big bang moment: Alex Krizhevsky’s groundbreaking deep neural network AlexNet, the first machine learning algorithm that could accurately label input images, revolutionizing the field of computer vision.
When OpenAI was founded in December 2015, Sutskever was just director of research, yet held a more senior position than co-chairs Altman and Elon Musk. That made sense at the time, since OpenAI was originally founded as a nonprofit dedicated to creating value for everyone, not shareholders, and putting “the good of all over self-profit.”
But since then, OpenAI has effectively become a for-profit company, in Altman’s words, “to cover the costs of our compute-intensive business. In the process, the company adopted a complex new for-profit structure that limited returns to investors, including Microsoft and Khosla Ventures, but retained control in the hands of a non-profit board of directors.
Altman said at the time that this complex governance was necessary to keep everyone on board. Recently, The Information reported that he sought to change OpenAI’s legal structure, paving the way for a controversial IPO.
Sutskever’s new commercial company, focused on secure superintelligence, will be based in Palo Alto, Silicon Valley, and Tel Aviv to best recruit the best talent.
“Our team, investors and business model are all aligned towards achieving SSI,” they wrote, pledging to “not be held back by administrative overhead or product cycles.”
But it wasn’t immediately clear from the statement how he and his two co-founders plan to create ASI with strong guardrails while also paying its bills and earning returns for investors. For example, it wasn’t clear whether ASI will also have a capped profit structure.
They said only that Safe Superintelligence’s business model was designed from the beginning to be “insulated from short-term commercial pressures.”
