Why does Nvidia dominate AI chips, and what is CUDA?

In the quarter that ended July 26, 2026, Nvidia reported $89.0 billion in data centre revenue, up 117% from a year earlier, and a gross margin of 75%. Gross margin is the share of each sales dollar left after the direct cost of making the product. A margin that high usually means customers feel they have few substitutes.

The size of the gap shows up when you line up the latest reports from the closest public competitors. The categories are not identical, so read this as a rough scale and not a market share calculation.

Company	Quarter ended	Reported line	Revenue
Nvidia	July 26, 2026	Data Center	$89.0 billion
Broadcom	August 2, 2026	AI semiconductors	$16.7 billion
AMD	June 27, 2026	Data Center, which also includes server CPUs	$6.7 billion

This table leaves out chips that companies build for themselves and do not sell, such as Google's TPUs and Amazon's Trainium, so it understates the competition Nvidia faces.

What CUDA is

A GPU contains thousands of small processors. Telling them what to do takes special code. CUDA, which Nvidia introduced in 2006, is the toolkit for writing it: a compiler, a runtime, debugging tools and many ready-made libraries that do common math very quickly. It only targets Nvidia hardware.

Nvidia's annual filing describes CUDA as the foundation of its software stack and says more than 7.5 million developers use CUDA and its other tools. It also says the company has spent more than $76.7 billion on research and development since its founding.

CUDA's early start compounded. Researchers wrote their code for it because Nvidia GPUs were what they had. Libraries were tuned for it. New staff learned it in school and on the job. Each step made the next person more likely to choose Nvidia, which is what people mean by "the CUDA moat".

The lead is more than one piece of software

Three other things reinforce it.

Systems and networking. A large model runs on many chips that must exchange data constantly. Nvidia sells the chips, the links between them (NVLink) and whole racks, so buyers get something already tuned to work together.
Supply. Nvidia's latest quarterly filing shows supply commitments of $279 billion, up from $119 billion a quarter earlier, mainly for memory and manufacturing capacity. In a shortage, whoever has booked the factory time can ship, and everyone else waits.
Availability. The filing says its platforms are offered by virtually every major server maker and cloud provider. Anything a team learns on one cloud carries over to another.

How deep the moat really is

It depends on what you do. In a December 2024 analysis, The Register concluded that the moat is very real for anyone writing low-level GPU code, because old CUDA code has to be ported and tuned by hand, and conversion tools such as AMD's HIPIFY still leave manual work.

Most people do not work at that level. PyTorch, the most common framework for building AI models, sits above the hardware layer and calls whichever backend is installed. Intel and AMD executives told The Register that most developers now program at that level. They have an obvious interest in saying so, but the direction is visible in what is shipping: AMD maintains PyTorch builds for its ROCm software, and Google has released TorchTPU, an open-source way to run PyTorch directly on its TPUs. Getting code to run is the easy part, though. Getting it to run as fast and as reliably as on Nvidia is where the remaining work tends to be.

Where Nvidia is exposed

Nvidia's own filings list the pressure points, and they are worth reading as risk disclosures.

Pressure	What the filing says
Customers building their own chips	Names Alphabet, Amazon, Microsoft and others among competitors with internal chip teams
Few very large buyers	Two direct customers made up 22% and 14% of revenue in the fiscal year ended January 2026
China	Its outlook for the current quarter assumes no data centre compute revenue from China
Supply concentration	Relies on foundries TSMC and Samsung, and on memory from SK hynix, Micron and Samsung

The largest buyers have the money and the motive to reduce their dependence, since every point of Nvidia's margin is a cost to them. That is why custom chips from Broadcom and AMD's GPU deals matter so much to the story. Nvidia, for its part, also invests in its customers, including Canada's Cohere, whose latest funding round counts Nvidia among its backers.

This article explains the company's position. It is not a view on whether its shares are a good or bad investment.

Try it: Pick an open-source AI project on GitHub and search its code for the word cuda. Count how many hits are a simple device setting and how many are custom kernels (hand-written GPU routines). The first kind moves easily to other hardware. The second kind is the moat.

Can a developer avoid CUDA today?

For many projects, yes, at least at the level of the code you write. If you build in PyTorch and use standard models, you may never touch CUDA directly. This is the common pattern:

import torch

# Picks whichever accelerator PyTorch can see
device = "cuda" if torch.cuda.is_available() else "cpu"
model = model.to(device)

AMD's ROCm build of PyTorch exposes its GPUs through that same torch.cuda interface, so code like this can often run unchanged. The harder cases are custom kernels, libraries only published for CUDA, and the newest performance tricks, which often reach Nvidia first.

So the practical answer depends on your stack. List the libraries you rely on, check which ones support your target hardware, then time one real job on each. A smaller bill per chip means little if the job takes twice as long or your team spends a month porting it. For a learner, Nvidia remains the path with the most tutorials and the fewest surprises, and PyTorch keeps most of what you learn portable.