Nvidia Faces New Challenges in the AI Compute Marketplace It Helped Build

For years, Nvidia has been the dominant force in the AI hardware industry, but the company has faced a challenging few months. According to Bloomberg, Nvidia’s stock has fallen about 15% from its May peak despite continued growth in projected revenue. Based on expected earnings, the company now trades at a valuation below that of the average S&P 500 company, meaning investors are paying less per dollar of Nvidia’s projected profit than for many other large U.S. companies.

Investment continues to pour into AI infrastructure, but much of that capital is now shifting toward memory chip manufacturers. During the same period, Micron, one of the world’s largest producers of DRAM memory chips, has nearly tripled in market value. As AI data centres continue to expand, memory has emerged as one of the industry’s biggest bottlenecks, replacing the GPU shortages that dominated discussions last year.

For many observers, the shift is surprising considering Nvidia’s technological leadership. Innovations have driven the company’s success. One of them is CUDA, the software platform that helped establish Nvidia GPUs as the standard hardware for AI development and that has enabled Nvidia’s rapid pace of GPU innovation. Nvidia’s processors remain among the most advanced and technically sophisticated products ever manufactured.

By comparison, the business model of companies like Micron is far simpler. They manufacture high-bandwidth memory (HBM) chips that move data rapidly between processors. While the technology itself has improved steadily over many years, soaring demand from AI infrastructure has dramatically increased the value of these memory products. Because supply has struggled to keep pace with demand, prices for high-bandwidth memory have risen sharply over the past year.

Industry data shows that spot prices for DRAM—the price customers pay on the open market rather than through long-term contracts—have climbed significantly since 2023. Rather than being driven by a major technological breakthrough, the increase reflects the industry’s underestimation of the amount of memory required to support the global expansion of AI data centres.

The trend looks very different for Nvidia’s products. Data from the compute marketplace Ornn indicates that the spot price for renting Nvidia H100 GPUs peaked at around $3.20 per hour in May, then steadily declined. As GPU availability has improved and more alternatives have entered the market, the cost of AI compute has gradually fallen.

Because Nvidia’s business is closely tied to demand for GPU computing, falling compute prices have created pressure on its valuation. Memory manufacturers, meanwhile, continue to benefit from rising DRAM prices as demand remains strong.

Wayne Nelms, co-founder and chief technology officer of Ornn, believes the situation largely comes down to supply and demand. He noted that companies including Google, Amazon, Microsoft, and OpenAI have all introduced their own AI processors to reduce dependence on Nvidia. While those custom chips may not outperform Nvidia’s latest hardware, they have increased competition and helped lower compute costs.

Nelms pointed out that while many companies are now designing their own AI accelerators, almost none are manufacturing their own DRAM memory. Until there is either a major technological breakthrough in high-bandwidth memory, a significant change in supply and demand, or new competitors enter the memory market, he expects current pricing trends to continue.

Ironically, Nvidia’s current position is largely the result of its own success. By demonstrating the enormous value of AI computing, the company helped create a rapidly expanding market that has attracted numerous competitors. As competition puts downward pressure on GPU prices, companies supplying supporting technologies such as memory are now benefiting from some of the strongest gains in the AI infrastructure sector.