NVIDIA solved the sim-to-real gap problem in robot training

Jim Fan, AI Director at NVIDIA, shared an impressive achievement by their team. Humanoid robots learned to walk and navigate in space without prior training in the real world.

All training was conducted exclusively in a virtual environment. After completing the simulation, the robots were immediately sent to perform tasks in the real world. And they handled them without any additional adjustment or adaptation. The most surprising thing about this process is the incredible compression of learning time. What would take 10 years in reality was condensed into just 2 hours of virtual training.

How did NVIDIA engineers manage to achieve such a result? First, the simulation has no physical limitations inherent to the real world. A robot can fall and get up even 1,000,000 times in a row without risk of breaking. In reality, each fall could lead to serious damage to expensive equipment.

Second, in a virtual environment, the flow of time can be significantly accelerated. The simulation has no “real-time” limitations – the process can be run at any speed, as far as computing power allows.

Third, the developers applied the method of parallel learning. In a virtual environment, many digital copies of the robot can be run simultaneously. And gather experience from all of them at once, which radically accelerates the accumulation of data for training.

Another unexpected discovery: modeling human-like movements did not require gigantic neural networks. A model with 1,500,000 parameters – not billions! – proved sufficient to reproduce the “subconscious mechanics” of the human body. This is several orders of magnitude smaller than modern language models.

The key challenge that NVIDIA engineers managed to overcome is known as the “sim-to-real gap” problem. Usually, robots trained in a virtual environment experience difficulties when transitioning to the real world due to imperfect simulation. However, the NVIDIA team managed to create such an accurate physics model in the virtual environment that robots were able to apply the acquired skills in reality without additional adjustments.