Open-source models push enterprises to buy inference, not just training clusters

SAN FRANCISCO — The enterprise AI stack is splitting in two. Training remains a hyperscale sport. Inference is becoming a capacity-and-evaluation problem that a competent IT shop can shop around.

Vendors that sold exclusive model access are being asked to compete with open weights wrapped in a safety and logging layer. The moat, buyers said, is now latency, eval harnesses, and the ability to prove a model did not roam into the wrong data.

That is not a smaller market. It is a different one. It rewards the clouds and chipmakers who can serve a billion cheap tokens without lighting the CFO on fire.