AI Infrastructure Shifts From Models to Memory, Networks and Chip Supply

| 2 Min Read
This week’s coverage tracks AI benchmarks, cloud CPUs, chip design models and the materials and optical links needed to scale data centers.

The next phase of AI infrastructure will be measured across more than accelerator speed. CPUs, memory bandwidth, chip design software, optical networking and access to manufacturing materials are all becoming part of the performance equation. This week’s Tom’s Hardware coverage follows those shifts, from early work on agentic AI benchmarks to Nvidia’s effort to reserve capacity across its supply chain.

Measuring AI workloads beyond the GPU

As more AI models run locally, familiar PC benchmarks leave important questions unanswered. Throughput and tokens per second are useful, but they don’t fully capture how an agentic workload uses a CPU. Tom’s Hardware CPU editor Jake Roach is beginning to define how those workloads might be measured, including a benchmark focused on agentic AI that runs without a GPU.

That work matters because agentic systems can put different demands on hardware than a single model response. The goal is to establish a way to measure CPU performance in this emerging workload category. The benchmark is still being explored, so the value of the piece lies partly in showing how the test is being developed, and what questions need answers before results can be compared.

A DGX Spark sitting on top of a graphics card.

(Image credit: Tom's Hardware)

For practitioners experimenting with local models, terminology can be another source of friction. A separate primer explains concepts including inference, quantization, context, model weights and KV cache, and connects those terms to how models are served and run. The reference is intended to grow as the publication’s reporting on local AI develops.

Vera enters cloud service

Nvidia’s Vera CPU is moving from a component associated with its NVL72 rack into standalone cloud instances. Oracle announced that Vera instances are generally available on OCI superclusters, placing the processor in a cloud setting where customers can access it independently of the familiar rack configuration.

Nvidia designed Vera with agentic AI workloads in mind. Tom’s Hardware has also tested a Vera server locally, comparing it with systems based on Intel Xeon 6 and AMD EPYC Turin processors. The testing found a substantial memory bandwidth lead over a dual Xeon 6 server. That result gives the cloud launch a useful hardware context: the question is not just where Vera is offered, but how its design characteristics map to real workloads.

Specialized models reach chip design

Cadence’s new Maestro suite takes a different approach to applying language models: it tunes them for electronic design automation flows and pairs them with the company’s agentic tools. The intended use is to automate parts of chip design, a field where generic model performance may not translate directly into useful engineering results.

Cadence says early comparisons with general-purpose AI models show up to 30% higher result quality and as much as five times the token efficiency on the same task. Those are company-reported results, and access is expected to begin with select customers in the fourth quarter of 2026. The figures point to the practical case for task-specific models: if they can produce better results while using fewer tokens, the workflow may become more efficient. The claims will need to be judged in the context of customer use and the design tasks involved.

Cadence diagram of AgentStack over five AI Super Agents, with the new Maestro AI models layer, the JedAI data and AI platform, and cloud or on-premises compute beneath.

(Image credit: Cadence)

Supply and networking become constraints

More AI systems require more than GPUs. Nvidia has announced billion-dollar deals with six suppliers to support production capacity across substrates, advanced packaging, semiconductor testing and optical engines. One focus is Ajinomoto Build-up Film, or ABF, a substrate material used in chip packaging. The agreements are designed to secure future capacity; in return, suppliers reserve production for Nvidia.

That arrangement reflects a less visible constraint in accelerator production. Chip output depends on the materials and processes around the silicon, including the substrates used to build circuit boards. As demand expands, access to those inputs can shape how quickly finished systems reach customers. Nvidia’s supplier deals are an effort to secure several links in that chain at once.

Data center networks face a related scaling problem. Copper connections are approaching their practical limits for bandwidth, prompting interest in photonic links and switching. PsiQuantum has shown an 8x8 optical switch prototype that it says loses less light than competing devices. The company plans to develop larger devices, with a potential path into the photonic switch market. The prototype is an early step, but it reflects how network design is becoming part of the broader effort to connect large numbers of AI accelerators.

Compute economics and software experiments

Anthropic’s S-1 filing offers a rare look at the economics and infrastructure behind a major AI lab as it prepares for an IPO. The company disclosed $800 billion in GPU and TPU contracts. It also said that most of its compute capacity is used for research and development, including model work, new features and testing, rather than serving customers.

That allocation complicates the assumption that all compute spending directly supports paid usage. The filing also indicates that Anthropic’s revenue per compute dollar spent is favorable, challenging the idea that the compute cost of generating tokens necessarily exceeds what users pay. The disclosures provide substantial material for analysis, while leaving some company details undisclosed.

AI tools are also showing up in an unexpected software workflow: converting console games to run natively on PCs. Recent projects use tools including Reverse Engineer Anything and Ghidra to help with decompilation and recompilation. The reporting examines how AI agents can work with a unified toolkit, while byte-for-byte checks help catch incorrect output. It’s a concrete example of AI-assisted development being applied to existing software, with verification playing a central role in keeping the process grounded.

The common thread across these developments is that AI progress increasingly depends on engineering the full system: workloads that use CPUs, models shaped for specific tasks, reliable access to production inputs, and networks that can move data at scale. For industry teams, the useful signal is where bottlenecks are shifting—and whether new benchmarks and prototypes can measure or relieve them.

Premium Subcription: $29
Tom’s Hardware Premium offers a full year of access for $29, or subscriptions from $7 per month. Subscribers receive daily news analysis, specialist coverage of the semiconductor industry and access to Bench, the publication’s benchmarking database.View Deal

The linked reports and Bench database are available through a Tom’s Hardware Premium subscription.

Source: [email protected] (Sayem Ahmed) · www.tomshardware.com

Comments

Please sign in to comment.