GPU Market Forces
This past week we saw select U.S. export restrictions eased letting Nvidia and AMD resume shipping GPUs to China. Nvidia placed an order for 300k H20’s from TSMC as China’s demand surges.
Intense demand from China paired with easier export avenues i.e. eBay’s export program and freight forwarding mean supply of GPUs is continually being syphoned from the domestic U.S. market. This is elevating the secondhand GPU market.
If you’re a smaller domestic AI company this could impact your ability to source cheap GPUs domestically and you may want to seek out hardware alternatives and lock in deals, especially during the summer months and the extended October holiday.
Decentralized Compute
The new Broadcom Jericho 4 AI router ASIC introduces a way for small data centers to scale their operation into a cluster, obviating the NVlink lock-in. While this is a relatively new development, it would enable a modular scale up strategy for startups that don’t want to go all in.
Vultr launched on-demand AMD Instinct MI300X rental this week. AMD seeks to compete with Nvidia but has not traditionally kept up. If they are able to break into the market as a cost player, it would provide a second supplier of HPC chips. Might be time to take a look at AMD’s stack.
Additionally, marketplaces like Hyperbolic are significantly lowering the costs of access vs large hyperscalers to H100s & H200s run in smaller, more agile data centers. marketplaces like these will be an opportunity for independent owners of hardware that seek to monetize it while not in use for it’s primary purpose.
Power Update
Google agreed to a rare demand-response power reduction deal during grid stress periods. The U.S. generation is far less than what we’re predicted to need, and while this is a sign that it’s not keeping up, the scrappier data centers that are willing to use edge generation may bring valuable uptime at the margins.
Leave a Reply