MENU

Selling shovels in the AI gold rush

Selling shovels in the AI gold rush

Feature articles |
By Nick Flaherty



Jensen Huang received a rockstar welcome in front of 11,000 people in a stadium in San Jose this week.

As CEO and founder of Nvidia, Huang has seen his company become the world’s third most valuable company worth $2.29tn.

And yet he is insistent that Nvidia is about selling the tools and infrastructure for artifical intelligence models such as transformers, despite being part of the paper that first described the technology. This focus on selling the shovels to the miners in a gold rush is a key and was clearly extended with week with a series of announcements.

“Nvidia doesn’t build chips we build data centres. AI is not a chip problem, it’s a software problem. We won’t invent AIs but we will manufacture AIs.” This is at the heart of its ‘AI foundry strategy’ to create the hardware and software infrastructure to supply AI models.

This infrastructure is the NIMs and NeMo/

The Nvidia Inference Microservice (NIM) is a microservice that packages up a model with all the software it needs to run anywhere, whether in the data centre, a laptop or a Jetson board, as long as there is an Nvidia GPU with Tensor cores and the CUDA software.

“The library of the future is a microservice as it is not just described in mathematics but in AI,” said Huang. “Libraries such as cuDNN and cuLitho in the future will be NIMs. We will make the NIMs so they are very performing.”

Nvidia is taking open source and popular models and packaging them into NIMs for developers to use. But developers can also package up their own models with the NeMo tool.

NeMo Retriever microservices let developers link AI applications to business data — including text, images and visualizations such as bar graphs, line plots and pie charts — to generate highly accurate, contextually relevant responses. With these retrieval-augmented generation (RAG) capabilities, enterprises can offer more data to copilots, chatbots and generative AI productivity tools to boost accuracy and insight.

Additional NeMo microservices are coming soon for custom model development. These include NeMo Curator for building clean datasets for training and retrieval, NeMo Customizer for fine-tuning LLMs with domain-specific data, NeMo Evaluator for analyzing AI model performance, as well as NeMo Guardrails for LLMs.

“We will create, partner, take open source for production quality pre-trained models and adapt them, fine tune them, guard rail, and surround them with NeMo services,” said Huang.

The NIM microservices can also be packaged in software containers such as Kubernetes and managed with orchestration tools.

This starts with the enterprise market as the AI foundry but expands out to the industrial world, and in particular humanoid robots. The latest Blackwell GPU array will be integrated with ARM cores into Thor chip that will be used in those robots and will run models in the NIM containers.

These models will be trained in a new tool called Issac Lab operating in the Omniverse digital twin virtual environment. Omniverse has been re-written form the ground up in the last two years to support this move.

Of course the real world makes an appearance, and the geopolitics of China, that manufacturing powerhouse, will be important.

“There are two things we will do right away [regarding China],” said Huang. “Make sure we understand the policies and second do everything we can to create resilience in our supply chain. For example our Blackwell system has 35,000 parts with 8 coming from TSMC

“The DGX data centre has 600,000 parts from all over the world and many are from China. This is my truth, as it is for the automotive industry and for the defence industry so the supply chain is complicated but I have every confidence the doomsday scenario is not likely to happen, what we can do relate to resilience and compliance.”

The GTC conference this week covered a wide range of different technologies highlighting the range .of operation of the company

If you enjoyed this article, you will like the following ones: don't miss them by subscribing to :    eeNews on Google News

Share:

Linked Articles
10s