Nvidia’s trillion-dollar market cap isn’t about your gaming PC | Intent, 0009

Intent is all about helping talent in tech become more intentional with their career by becoming more informed, more fluent, and more aware about the goings-on within tech and adjacent industries.

On today’s Intent: a deep dive on the computing company in the spotlight, Nvidia.

Job searching? We’re built to help

Check out CareerMakers, by Free Agency, the job search prep program made for tech people, by tech people.

We’ve helped candidates negotiate over $300M in compensation since 2019 — supercharge your job search and get started now.

What the heck is Nvidia up to?

How did a company that was previously known for making graphics cards for video gamers find itself to be the hottest commodity in tech? Chipmaker-giant Nvidia was recently valued at $1 trillion thanks to their role in the AI explosion, joining Apple, Microsoft, Google, and Amazon as the only American companies to pass that threshold.

Why are they so pivotal to AI development? Training powerful generative AI models and processing these large datasets requires a lot of computing power, specifically from GPUs. ChatGPT, for example, was trained on thousands of Nvidia GPUs.

Nvidia controls 80% of the GPU market. It’s not just the size of the market share, but their quality of GPUs available — specifically, the NVIDIA H100 Tensor Core GPUs. These chips are the ”workhorse” for AI training thanks to their expansive memory bandwidth of over 2TB/s, enabling the breakdown of massive datasets required for LLM and generative AI development. They’re just flat-out faster and more powerful than the competition that AMD and Intel present, with the large majority of AI papers citing Nvidia chip usage according to the State of AI Report Compute Index.

Let’s take a closer look at what they’ve been up to lately.

Nvidia has new products coming out of its ears

Nvidia is expanding its product offerings seemingly every day, including at the recent SIGGRAPH conference highlighting new innovations in generative AI, OpenUSD, and more. (If you feel like binge-watching full demos from the SIGGRAPH conference, check out this dedicated YouTube playlist.)

They’ve recently announced:

A new line of superchips

The Grace CPU Superchip line is the first data center CPU developed by the company. The new superchip is built specifically to solve scaling problems plaguing AI models and hyperscale workloads, and is powered by their NVLinkChip-2-Chip, enabling 900GB/s bidirectional bandwidth to either another CPU or the Nvidia Hopper GPU. It’s super technical, so we asked ChatGPT to “explain like I’m 13.” Here’s what it said:

The NVIDIA Grace CPU is like the super-smart brain of a futuristic robot, built specifically for heavy thinking tasks like scientific research or creating virtual worlds. It uses the Armv9-A architecture, which is like using a super-efficient set of instructions to get things done, rather than the more complex but powerful x86 architecture used in most computers. Think of it as building with streamlined LEGO blocks instead of a complex jigsaw puzzle.

What makes this Grace CPU special is how it talks super-fast with other brains (CPUs) or visual experts (GPUs) using something called NVLink. Imagine a lightning-fast secret handshake that lets them share ideas instantly. It's designed to work in a way that's like having a supercharged engine that's also fuel-efficient, making it perfect for tasks that need a lot of brainpower but also care about saving energy.

Thanks, ChatGPT. For the technical folks, here’s an in-depth blog on all the specs if you want to learn more.

AI workbench

AI Workbench is a new development platform enabling developers to build and test their own generative AI models and LLMs from scratch from a PC or workstation. As part of Nvidia's effort to democratize AI development, the product automatically creates AI project environments, simplifies the selection process of foundation models, and enables fine-tuning. Then, it allows users to scale models to any cloud server and data center. It’s a lot — just watch their announcement video.

Currently, developing accurate powerful models with proprietary data is a huge challenge, and requires much deeper knowledge from developers to manually design architecture, train and fine-tune models, and then deploy them to production. But with AI Workbench, this process could open up development in a huge way.

The CloudXR Suite

Nvidia’s CloudXR Suite is built to power the streaming of (extended reality) content to untethered devices. It leverages RTX-powered servers with GPU visualization software to run complex VR and AR experiences from a scalable remote server across 5G and wifi networks. In other words, VR and AR apps can run straight from the cloud with nearly-the-same quality as a native device, which could mean that wireless, user-friendly devices might finally become viable.

Nvidia Picasso

Picasso is Nvidia’s “foundry for custom generative AI,” providing developers with an easy-to-use platform to build, customize, accelerate, and deploy their own foundation models for image, video, 3D, and 3D HDRI assets. Instead of having to work from scratch, Picasso streamlines model training, optimization, and inference on the Nvidia DGX Cloud (Nvidia’s AI-Training-as-a-Service solution).

They’ve already reached a partnership with Shutterstock to generate fully-licensed high-fidelity 3D assets for the platform, along with partnerships with Getty Images and Adobe.

The Omniverse platform

Omniverse, their low-code/no-code development platform, does a lot, including enabling creators to create 3D workflows and applications through augmented AI. It’s been around for a while but has had some new updates announced recently. Namely, it offers new foundation applications for developers to enhance their 3D pipelines with OpenUSD workflows. We wrote a lot about OpenUSD in a previous edition of Intent — check it out.

So, what’s next for Nvidia?

Even beyond all of these new tools, Nvidia seems dedicated to becoming the face of AI development and enablement. They recently led a $1.3B investment round for Inflection, which promises to build a more “personal” AI than the rest. Nvidia and Inflection are collaborating to train models on one of the largest AI training clusters in the world — made up of more than 22,000 Nvidia H100 GPUs.

They also recently secured a $5B order from Chinese tech giants like Alibaba, ByteDance, and Tencent (possibly to get ahead of the Biden administration’s new export controls).

Their CEO recently said that their chips will “reinvent the computer itself.” With the new Grace superchip line, he stated that “You could take just about any large language model you like and put it into this and it will inference like crazy. The inference cost of large language models will drop significantly." In other words, the speed and cost of running large language models and getting outputs is about to go down — by a lot.

So, is Nvidia about to take over the computing hardware industry as we know it? Or will competitors like Intel and AMD finally make good on their promise of catching up? AMD recently released their own AI chip, the MI300x, which compares pretty favorably to the Nvidia H100. But with Nvidia’s market share and riding of the hype cycle, it’s a tough hill to climb.

Think a friend could use a dose of Intent?

Forward this along to your least-informed friend in tech. We love the attention. ;)