French AI Startup ZML Launches Free Inference Platform for Multi-Chip AI Performance

Nvidia’s dominance in the AI hardware market remains strong, but competition is steadily increasing as new alternatives continue to emerge from multiple directions.

French AI startup ZML, which has received backing from Turing Award recipient Yann LeCun, has introduced new inference-performance software designed to run a wide range of open-source large language models across multiple hardware platforms. The software supports chips from Nvidia, AMD, Google’s TPU ecosystem, Apple Metal, and Intel Arc, among others.

Called ZML/LLMD, the newly released large language model inference server is designed to remove the barriers separating different hardware ecosystems. According to ZML founder Steeve Morin, the goal is to enable organisations to run AI workloads across various chips at their maximum possible performance—and in some cases even exceed existing performance benchmarks.

Morin explained that as artificial intelligence becomes increasingly embedded in both business operations and everyday life, inference—the process of handling user prompts—has grown more important than model training. However, software limitations and hardware-specific architectures continue to create fragmented environments that often lock customers into individual vendors.

Delivering high-performance inference across numerous chip architectures is not only a significant technical achievement. Still, it could also reshape the AI hardware market at a time when organisations are becoming increasingly concerned about the growing cost of deploying artificial intelligence.

ZML aims to give enterprises and cloud providers greater flexibility by allowing them to combine different types of AI processors, including hardware that may be less expensive or consume less power. Morin said the company’s objective is to return control to customers, enabling them to build AI infrastructure that delivers genuine efficiency improvements while making AI adoption more accessible.

According to Morin, this type of software support could particularly benefit emerging AI chip manufacturers, many of which are based in Europe. He pointed to companies including Axelera, Fractile, Kalray, OLIX, Q.ANT, SiPearl, SpiNNcloud, and VSORA as examples. More important than their geographic location, he said, is ZML’s ability to collaborate with these companies on technologies that have not previously been achieved elsewhere in the industry.

Despite helping broaden hardware competition, Morin stressed that he remains optimistic about Nvidia’s future. He said ZML maintains a strong relationship with the AI chip leader, which has already been preparing extensively for the growing importance of AI inference.

Inference has attracted enormous industry investment, with many referring to the current trend as the “inference gold rush.” As a result, ZML faces competition from companies such as Baseten, recently valued at $13 billion, Inferact, developed by the creators of the open-source project vLLM, and RadixArk, the commercial company behind SGLang.

Although both vLLM and SGLang compete with LLMD in certain areas, Morin said ZML’s long-term ambitions extend well beyond inference software alone. He explained that the company has already reached the stage where it is collaborating on silicon design itself. Morin also credited ZML’s compact workforce of around 20 employees for enabling the Paris-based startup to move rapidly while preparing additional product releases.

The company’s relatively small team has also benefited from substantial financial backing. Drawing on his previous experience as Vice President of Engineering at Zenly, which Snapchat acquired for a nine-figure sum in 2017, Morin successfully raised $20 million from investors including Harry Stebbings’ 20VC, >commit, AALVC, Drysdale Ventures, Xavier Niel’s Kima Ventures, Kindred Capital, LocalGlobe, and Puzzle Ventures.

Unlike ZML’s first publicly released project—its inference-focused machine learning framework introduced in 2024 and updated in March—ZML/LLMD is not open source. Nevertheless, the company is launching the software as a free product to understand better how customers use it. Morin said he prefers to measure adoption first before deciding how and where revenue can be generated, rather than limiting the company’s growth by introducing monetisation too early.

It is still too soon to determine when ZML/LLMD could transition into a paid offering or how widely it will ultimately be adopted. However, the startup’s investor list suggests that several influential figures within the AI industry are closely following its progress. Those supporters include Dagger and Docker founder Solomon Hykes, Hugging Face founders Clément Delangue and Julien Chaumond, and Yann LeCun through AMI Labs. Morin believes their support also demonstrates that Europe’s AI startups can successfully build world-class technology without relocating abroad, adding that he could not have built ZML anywhere other than Paris.