Nvidia boosts AI training for humanoid robots
Cette publication existe aussi en Français
Nvidia is providing its AI services and models to robot manufacturers for the next generation of humanoid robots by most of the leading developers.
Nvidia has developed new AI microservices and frameworks for humanoid robots as part of its NIMS offering launched earlier this year along with the Jetson Thor platform running the AI on a variant of the Blackwell GPU.
These will be used by 1x, Boston Dynamics, ByteDance Research, Field AI, Figure, Fourier, Galbot, LimX Dynamics, Mentee, Neura Robotics, RobotEra and Skild AI who have all signed up to a new Humanoid Robot Developer Programme that provides early access to a range of new tools including the Project Gr00t foundation models..
Nvidia company has also developed an orchestration service called Osmo for running multi-stage robotics workloads, and an AI- and simulation-enabled teleoperation workflow that allows developers to train robots using small amounts of human demonstration data.
“The next wave of AI is robotics and one of the most exciting developments is humanoid robots,” said Jensen Huang, founder and CEO of NVIDIA. “We’re advancing the entire NVIDIA robotics stack, opening access for worldwide humanoid developers and companies to use the platforms, acceleration libraries and AI models best suited for their needs.”
NIM microservices provide pre-built Docker containers with Nvidia inference software to reduce deployment times from weeks to minutes. Two new AI microservices will allow roboticists to enhance simulation workflows for generative physical AI in Isaac Sim, a reference application for robotics simulation built on the Omniverse platform.
The MimicGen NIM microservice generates synthetic motion data based on recorded teleoperated data from spatial computing devices like Apple Vision Pro. The Robocasa NIM microservice generates robot tasks and simulation-ready environments in OpenUSD, a universal framework for developing and collaborating within 3D worlds.
Osms is a cloud-native managed service that allows users to orchestrate and scale complex robotics development workflows across distributed computing resources, whether on premises or in the cloud.
This vastly simplifies robot training and simulation workflows, cutting deployment and development cycle times from months to under a week. Users can visualize and manage a range of tasks — like generating synthetic data, training models, conducting reinforcement learning and implementing software-in-the-loop testing at scale for humanoids, autonomous mobile robots and industrial manipulators.
Training foundation models for humanoid robots requires an incredible amount of data. One way of capturing human demonstration data is using teleoperation, but this is becoming an increasingly expensive and lengthy process.
Humanoid robot AI workflow
A teleoperation reference workflow, demonstrated at the SIGGRAPH computer graphics conference, allows researchers and AI developers to generate massive amounts of synthetic motion and perception data from a minimal amount of remotely captured human demonstrations.
First, developers use Apple Vision Pro to capture a small number of teleoperated demonstrations. Then, they simulate the recordings in Isaac Sim and use the MimicGen NIM microservice to generate synthetic datasets from the recordings.
The developers train the Project GR00T humanoid foundation model with real and synthetic data, enabling developers to save time and reduce costs. They then use the Robocasa NIM microservice in Isaac Lab, a framework for robot learning, to generate experiences to retrain the robot model. Throughout the workflow, NVIDIA OSMO seamlessly assigns computing jobs to different resources, saving the developers weeks of administrative tasks.
Fourier, a general-purpose robot platform company, sees the benefit of using simulation technology to synthetically generate training data.
“Developing humanoid robots is extremely complex — requiring an incredible amount of real data, tediously captured from the real world,” said Alex Gu, CEO of Fourier. “NVIDIA’s new simulation and generative AI developer tools will help bootstrap and accelerate our model development workflows.”
Nvidia AI supercomputers are used to train the models while the Isaac Sim enables the robots to learn and refine their skills in simulated worlds, and the Jetson Thor humanoid robot computers to run the models.
“Boston Dynamics and Nvidia have a long history of close collaboration to push the boundaries of what’s possible in robotics,” said Aaron Saunders, chief technology officer of Boston Dynamics. “We’re really excited to see the fruits of this work accelerating the industry at large, and the early-access program is a fantastic way to access best-in-class technology.”
Developers can join the Nvidia Humanoid Robot Developer Programme now to get access to Osmo and Isaac Lab, and will soon gain access to NIM microservices.
If you enjoyed this article, you will like the following ones: don't miss them by subscribing to :
