Sector 4 · 7 sub-themes
Compute & Infrastructure
OpenAI moved to build its own silicon. With Broadcom and Celestica, it unveiled "Jalapeño," an inference chip designed around OpenAI's own LLM workloads, not a general accelerator. Hock Tan confirmed first silicon samples are in hand and under test, with deployment expected at Microsoft. The intent is explicit: reduce dependence on Nvidia and own the full stack. Compute scarcity is now the industry's binding constraint. Google capped Meta's Gemini access — a raw capacity shortfall, not a pricing fight. Meta's answer was to launch "Meta Compute" to rent out its own excess capacity, sending the stock up roughly 9% while analysts warned of margin dilution. SK Telecom went bigger still, committing ~$653 billion to 15 GW of Korean data-center capacity by 2035. Nvidia's grip on rentals persists. B200 spot rates hit $5.98/GPU-hr, 131% above H100, yet only one provider has live capacity — supply stays severely constrained. Six-year-old H100s still fetch ~$3/hour, undercutting the obsolescence narrative. On efficiency, the fight is inference cost. DeepSeek's DSpark speculative decoding claims 51–406% throughput gains on V4. Nvidia open-sourced DFlash, a block-diffusion model promising up to 15× speedups via vLLM. KV-cache capacity is now a hardware design lever. Separately, Nvidia's 45°C warm-water closed loop aims to eliminate nearly all onsite water use — as much PR against AI's water critics as engineering.
OpenAIBroadcomCelesticaNvidiaMicrosoftHock TanGoogleMetaAWSSK TelecomDeepSeekDellcustom AI inference chipsGPU compute scarcityLLM inference optimizationAI infrastructure investmentspeculative decoding throughputKV cache economicsGPU rental pricingdata center coolingAI chip supply chain verticalizationhyperscaler compute monetization
4.1
OpenAI and Broadcom unveil Jalapeño, OpenAI's first custom AI chip
- OpenAI and Broadcom jointly unveiled "Jalapeño," OpenAI's first custom AI chip — officially described as an "Intelligence Processor" — architected specifically around OpenAI's LLM inference workloads rather than as a general-purpose accelerator. [3][2][1]
- Broadcom CEO Hock Tan confirmed that OpenAI has already received first silicon samples of the Jalapeño chip and is actively testing how the silicon handles running AI workloads. [5]
- The chip is expected to be deployed with Microsoft, reflecting OpenAI's broader push to build out its own AI infrastructure and reduce dependence on Nvidia. [4]
- The chip was developed in partnership with Broadcom and Celestica, with OpenAI positioning the move as an expansion beyond consumer products into AI infrastructure. [6][2]
OpenAIBroadcomJalapeñoHock TanMicrosoftCelesticaNvidiacustom AI chipLLM inference hardwareAI infrastructuresilicon testingreducing Nvidia dependenceIntelligence ProcessorAI accelerator development
Sources
- [1]CNBC — OpenAIandBroadcomon Wednesday unveiled their debut custom chip, called Jalapeño, marking theChatGPTmaker's first entry intoartificial intell★news
- [2]CNN — OpenAI on Wednesday announcedits first custom AI chipina step towards expanding beyond consumer products to become a player in AI infrastruc★news
- [3]{OpenAI, Broadcom} — OpenAI and Broadcom (NASDAQ: AVGO) today unveiled Jalapeño, OpenAI’s first Intelligence Processor: an accelerator architected around OpenAI’★news
- [4]wallstengine — OpenAI and Broadcom $AVGO unveiled Jalapeño, a custom AI accelerator built for LLM inference. The chip is expected to deploy with Microsoft ★twitter
- [5]wallstengine — $AVGO Hock Tan: OpenAI has received the first samples of the chip, called Jalapeno, and is testing how the silicon handles running AI worklo★twitter
- [6]gudanglifehack — OpenAI Unveils Jalapeño, Its First Custom AI Chip for ChatGPT OpenAI revealed Jalapeño, a custom AI chip built with Broadcom and Celestica, twitter
4.2
DeepSeek DSpark Speculative Decoding Boosts V4 Flash Inference Speed
- DeepSeek published DSpark, a speculative decoding system designed to accelerate live DeepSeek V4 serving, claiming throughput improvements of 51% to 406% under stricter serving conditions. [1]
- Multiple users running DeepSeek V4 Flash DSpark on 2× NVIDIA DGX Spark units report achieving approximately 60–66 tokens/second single-stream throughput, representing roughly 1.5× faster decode than the MTP-1 baseline and a ~50% improvement over prior configurations. [2][3][4]
DeepSeekDSparkDeepSeek V4DeepSeek V4 Flash DSparkNVIDIA DGX Sparkspeculative decodinginference throughput optimizationLLM serving accelerationtoken generation speedon-device model deploymentdecode latency improvement
4.3
Google caps Meta's Gemini access due to compute capacity shortage
- According to the Financial Times, Google placed limits on Meta's use of its Gemini AI models after Meta sought more computing capacity than Google could provide — described explicitly as a capacity constraint, not a pricing or licensing dispute, meaning even a major enterprise customer could not obtain the AI compute it requested. [4][5][6][7][8][9]
- Meta responded to compute constraints by announcing plans to launch its own cloud business to sell excess AI compute capacity, a move that caused Meta's stock to surge approximately 9% and was characterized by analysts as easing the biggest overhang on the stock. [1][2][3]
GoogleMetaGeminiFinancial TimesAI compute capacity constraintscloud infrastructureenterprise AI accessMeta cloud business launchGoogle Gemini licensingAI infrastructure investmentstock market reactionhyperscaler competition
4.4
KV Cache Optimization and LLM Inference Infrastructure Developments
- NVIDIA has open-sourced DFlash, a block diffusion model designed to accelerate autoregressive LLM inference by up to 15× while integrating with the vLLM serving framework. [1]
- IBM Research, Red Hat, and India's NxtGen Cloud Technologies found in recent experiments that serving AI models with llm-d can boost inference performance, though the specific magnitude of improvement is not fully stated in the available source text. [2]
- Storage Review highlighted a Dell PowerEdge XE7740 configuration intentionally removing 16 DIMMs — framed around the tradeoff between DRAM cost (noted at approximately a quarter million dollars at list price) and how much KV cache capacity that hardware can provide for LLM serving. [3]
NVIDIADFlashvLLMIBM ResearchRed HatNxtGen Cloud Technologiesllm-dStorage ReviewDellPowerEdge XE7740LLM inference accelerationblock diffusion modelsopen-source AI serving frameworksKV cache optimizationDRAM cost tradeoffsautoregressive inferencedistributed inference performanceAI hardware configurationmodel serving infrastructure
4.5
Nvidia GPU rental pricing, cooling technology, and compute infrastructure trends
- B200 spot GPU rental prices stand at $5.98/GPU-hr — 131% above H100 rates — yet only one cloud provider has B200 capacity live, with one more holding reservations, indicating supply remains severely constrained despite expectations of relief. [9]
- H100 chips, described as six years old, are still commanding approximately $3/hour in rental markets, with a payback period of roughly one year, indicating legacy Hopper hardware continues to generate strong returns. [10]
- Analysts note that recent rental pricing for H100/H200 Hopper GPUs may signal that market assumptions about rapid obsolescence of that generation are more extreme than current pricing supports. [7]
- B200 compute rental pricing is identified as a key variable disrupting forward margin models, with practitioners noting that the number "keeps breaking the entire equation" when attempting to forecast profitability. [8]
- RAM (DRAM) prices have roughly doubled since early 2025, with DDR5 reportedly rising approximately 90% in a single quarter, compounding AI infrastructure cost pressures alongside GPU pricing. [6]
- Nvidia is offering startup customers the option to exchange compute access for a revenue-share arrangement rather than upfront payment, reflecting a new commercial model for GPU access. [1]
- Nvidia has developed a 45°C warm-water liquid cooling system designed to address thermal management challenges in high-density AI data centers, with the approach enabling direct liquid cooling of the largest AI machines without chilled water infrastructure. [2][3]
- Nvidia's new cooling technology is reported to reduce data center water consumption, though analysts note that AI's total water footprint extends beyond cooling efficiency gains at individual facilities. [4][5]
NvidiaB200H100H200HopperDDR5DRAMGPU rental pricingAI compute costsB200 vs H100 market ratessupply constraintslegacy GPU returnsliquid cooling technologyDRAM price inflationrevenue-share compute accessAI infrastructure economicsdata center water consumption
4.6
Nvidia warm-water cooling system eliminates data center water use
- Nvidia announced a warm-water closed-loop cooling system that eliminates "pretty much all" onsite water use in data centers, marking a significant shift from traditional evaporative cooling methods that consume large volumes of fresh water. [2][3][6][8]
- The cooling breakthrough involves operating with water at up to 45°C, a threshold Nvidia describes as enabling full liquid cooling of its highest-density AI systems without requiring chilled or evaporative water infrastructure. [1]
- According to the Manhattan Institute, as cited by Nvidia, data center water consumption figures are frequently miscontextualized; Nvidia is actively promoting this framing to counter public concern about AI's water footprint. [7]
- Axios contextualized data center water use with a comparative chart, offering scale against other industrial water consumers to provide broader perspective on the AI industry's actual consumption share. [9]
- Remarks attributed to Jeff Bezos in a Paris speech — suggesting humans should reduce personal water consumption because AI data centers require it — were circulated widely and fact-checked by Snopes, which examined whether Bezos actually made that argument. [4][5]
NvidiaManhattan InstituteAxiosJeff BezosSnopesdata center cooling innovationwater consumptionliquid cooling systemsAI infrastructure environmental impactevaporative cooling alternativeswarm-water cooling technologyAI water footprintfact-checking
4.7
SK Telecom Plans 15 GW AI Data Center Expansion; Meta Launches Cloud Compute Business
- SK Telecom has announced a plan to build 15 GW of AI data center capacity across South Korea by 2035, with the investment reportedly totaling approximately $653 billion, representing one of the largest national AI infrastructure commitments announced by a single telecom operator. [1][2][3]
- Meta has launched "Meta Compute," a new cloud business designed to sell AI computing power to external customers beyond its own internal advertising workloads, marking a significant expansion of Meta's business model into the cloud infrastructure market. [4][6]
- According to Bloomberg, Meta's move into selling AI computing power externally signals a broader strategic shift in how large technology companies are monetizing their data center investments, with Meta planning to offer AI infrastructure as a service to third parties. [6][7]
- A McKinsey & Company analysis of colocation data centers describes an accelerating "infrastructure race" behind AI, with colocation facilities emerging as critical enablers of the rapid scaling of AI compute capacity globally. [7]
- Coverage from CIO.com notes that Meta's and Oracle's recent data center moves collectively illustrate shifting economics in the data center sector, where hyperscalers are increasingly moving to commercialize excess or purpose-built AI compute capacity externally rather than reserving it solely for proprietary workloads. [5]
SK TelecomMetaMeta ComputeOracleMcKinsey & CompanyBloombergCIO.comAI data center infrastructurecloud computing expansionAI compute capacityhyperscaler monetizationcolocation facilitiesnational AI investmentinfrastructure as a servicetelecom AI strategy