The Neoclouds vs Hyperscalers OCP panel discussion didn't just review the current state of cloud infrastructure—it revealed a fundamental, ongoing economic restructuring driven entirely by AI workloads. As a participant in that discussion, FarmGPU and its co-panelists confirmed that the decades-old assumptions about data centers, hardware margins, and what it means to be a "cloud provider" are being tossed out.
If you’re in hardware, cloud, or enterprise IT, here’s why the AI gold rush is an inversion of everything you thought you knew.
The New Center of the Universe: Tokens, Not Transistors
The most profound shift is the inversion of cloud economics. Traditionally, differentiation and margin were found in the infrastructure layer. The panel consensus confirms that’s over.
While GPUs are the new essential resource, they are rapidly becoming commoditized. As one of the panelists put it: "Today it's really famine for a lot of the hardware vendors. Nvidia is sucking all the margins."
The real value is migrating up the stack to whoever can most efficiently deliver tokens per second to the end application.
- Infrastructure is now the cost of entry, not the differentiator.
- The new margin opportunities are in scale-up networking (delivering a massive 30x performance gain compared to the mere 2x from a new chip generation), specialized storage for multimodal workloads, and complete solution delivery.
- As Andy (Cerebras) summarized the shift: "The margin really is... in delivering value all the way to users where they are."
Why "Neoclouds" Are Thriving (and Not Competing with Hyperscalers)
The AI boom isn't a winner-take-all scenario for the existing giants. Instead, market fragmentation is creating entirely new players—dubbed "Neoclouds"—who are exploiting structural inefficiencies the hyperscalers are simply too big to care about. As a leading Neocloud provider, FarmGPU’s CEO, highlighted the critical role of arbitrage:
- Geographic Arbitrage: Hyperscalers only build massive 100MW+ greenfield facilities. This leaves an entire market of 1MW to 10MW regional data centers "off the radar." In the words of FarmGPU's JM: "There are a lot of regional data centers... This is off the radar of the hyperscalers. It's so small now... But there's hundreds of millions of dollars to be made by the Neoclouds in these smaller pockets."
- Speed Arbitrage: A Neocloud can deploy capacity in 30 days versus the lengthy planning cycles of a major hyperscaler.
- Stranded Capacity Arbitrage: Leveraging existing facilities that cost "$0/MW because they're already there" offers radically different economics than a new build.
The Survival Criterion: For Neoclouds, long-term success isn’t about deployment speed, but about customer lock-in duration. Those securing long-term enterprise or sovereign deals will be the ones that last.
The Capital vs. Power Misdiagnosis
What is your true limiting constraint? Most players are misdiagnosing it.
While the industry obsesses over power efficiency, the panel’s analysis showed a non-obvious truth: for the vast majority of deployments, capital constraints matter far more than power efficiency.
- A 20% improvement in performance per watt only translates to a roughly 4% difference in Total Cost of Ownership (TCO) when a GPU has a 5x markup.
- This is why most Neoclouds optimize for performance per dollar, not performance per watt.
Your limiting constraint defines your architecture. Most players, like many Neoclouds, optimize for capital efficiency to deliver a better ROI.
Workload Diversity Guarantees Fragmentation
Perhaps the most important theme is that the sheer diversity of AI use cases makes a monopoly, or even a dominant vertical stack, impossible. No single solution optimizes for all AI workloads:
- Latency: Do you need a "race car" of tokens per second for real-time agents, or a "bus" for overnight batch processing?
- Modality: Text models have different needs than image, video, or robotics models.
- Use: Training, fine-tuning, inference, and new agentic workflows all require different architectures.
This diversity guarantees that hyperscalers, Neoclouds (like FarmGPU), and vertically integrated vendors will all continue to coexist—they are simply solving fundamentally different optimization problems based on the end user's need.
The Verdict on Open vs. Closed: It's About the Timeline
The debate over vertical integration (closed) versus an open ecosystem isn't philosophical; it's a trade-off between speed and optionality over a given timeline:
- Vertical Integration is faster and delivers massive, simultaneous performance gains across the stack. It makes sense early in a technology’s lifecycle when speed is paramount.
- Open Ecosystems become more beneficial as the technology matures and the timeline extends. They offer vendor lock-in avoidance, easier developer hiring, and the ability for operators to customize their environments. This pragmatic approach is key to Neocloud operations. As FarmGPU’s JM put it, the use of open source switches means: "The fact that we could just take Sonic out of the box, we don't have vendor lock in... if we have a bug ourselves and we want to fix it, we can get into the code and fix it. And that's one of the big reasons about open source software. You're not locked in."
The Bottleneck Has Shifted to Communication
The final, critical architectural takeaway is that the bottleneck has moved from compute to communication and whoever solves scale-up networking efficiently captures disproportionate value. The new driver of performance is networking topology, not chip improvements.
This isn't just a cycle; it's a paradigm shift. As Andy concluded, "When the internet started on dialup and got to broadband, we didn't just have more of the same. We had fundamentally new billion and trillion dollar industries."
AI is doing the same to cloud infrastructure. The industry is not just getting bigger; it's getting fundamentally different.