Sutskever announced the end of the era of AI model scaling

Ilya Sutskever gave an interview to Dwarkesh Patel. Sutskever states: “The era of model scaling is over: we are returning to the era of research”. Simply adding more computational power no longer works. New discoveries are needed, not just more money and chips.

The bottleneck of modern models — this is still generalization. This is precisely where artificial intelligence critically lags behind humans. And this is the main obstacle on the path to AGI.

Sutskever directly says that high scores on benchmarks — this is to some extent a mirage. Real quality lags behind because models are overtrained on tests during reinforcement learning. And researchers only aggravate the problem by stamping out identical benchmarks and training cycles.

And here a mystery appears. Sutskever says he has ideas about what current models lack, but doesn’t share them. Although hints slip through. Of course, at his company they are developing methods based on continuous learning and dynamic reward functions right now.

The forecast sounds vague: AGI can come in both 5 and 20 years. But his recipe for humanity’s survival sounds radical. The most reliable way not to remain useless and maintain control — merging with artificial intelligence.

It turns out, one of the creators of ChatGPT announces that the era of simple scaling is over. Benchmarks lie. And the only path for people — to become cyborgs.