AI infrastructure budgets often put GPUs at the center. But each model response can also create data that must be logged, cached, backed up or retained for later use. As inference becomes a larger share of AI activity, this accumulating footprint makes storage a recurring operating cost, not a small add-on to compute.

The distinction between training and inference helps explain why the budget can drift. A training run has a defined start and finish. Inference keeps running as users ask questions, generate images and produce video. Each request may leave behind data that persists after the model has delivered its answer.
One short video can leave a large data trail
AI-generated video illustrates the gap between what a user receives and what an organization may need to store. WD’s internal measurements found that producing an eight-second clip—delivered as a 3.97MB, 720p file—generated more than 500MB of associated back-end data. That included logs, cached context, intermediate renders, backups and safety records. The retained footprint was more than 130 times the size of the video file.
The number will vary by workload and system design, but the underlying planning question is consistent: what data accompanies each generated result, and how long does the organization keep it? For enterprises generating longer or higher-resolution video, the article says the back-end footprint can reach hundreds of gigabytes per clip.
Demand growth adds pressure. McKinsey projects AI inference demand will grow at a 35% compound annual rate through 2030, compared with 22% for training. It estimates inference will account for more than 40% of total data center demand by the end of the decade.
A WD-sponsored IDC survey of 763 IT and business decision makers points to data growth already underway: nearly 95% said generative AI had increased their data volumes over the previous 12 months, and about 61% reported growth of at least 25%.
Storage affects whether compute stays productive
The storage question is not only about the cost of keeping files. If data can’t reach a GPU when needed, expensive compute may sit idle. A senior product manager at a hyperscale cloud service provider, quoted in the report, described storage capacity as a way to keep GPUs busy: “The biggest challenge is to ensure that the GPUs are not idle, waiting for data to show up—hours of GPU idle time becomes very expensive, very fast.”
That link between storage and compute efficiency can make storage spend harder to spot in a budget. Compute purchases often follow visible training cycles, product launches and hardware refreshes. Retained data accumulates on a different timeline. Logs, inference results and synthetic outputs don’t vanish when a compute project winds down, and organizations may keep them for longer as their potential future use grows.
Data once treated as archival may become useful again as input for retrieval-augmented generation, or RAG. The WD-IDC survey found that 96% of respondents expect to need faster archive retrieval to support inference or RAG. That expectation points to a change in how teams may need to think about older data: some of it must be accessible enough to feed new workloads, even if it isn’t used constantly.
Budget for the data lifecycle, not just the generation step
Consider a retailer producing product videos or an insurer processing claims documents through a RAG pipeline. Compute costs may be forecast per video or query. Storage costs build with each item processed and may persist after the task is complete. A budget that treats storage as negligible can miss this compounding effect until operating costs rise or retrieval slows.
The article warns that, over 12 to 18 months, storage that began as a rounding error could become a significant share of an AI program’s total cost—just as leaders consider whether to expand it. The timing matters: a storage constraint can surface as an operational problem rather than as a clearly visible line item alongside accelerator purchases.
Planning across the data lifecycle means matching storage performance and capacity to how information is used. Frequently accessed data may need fast storage, while larger volumes of retained material can sit in warmer or colder tiers. For data that must be kept and revisited, the article identifies hard disk drives as a cost-effective, high-capacity option with an advantage in cost per terabyte. The appropriate mix depends on access needs, power use, reliability, density and the total cost of operating the system.
That architecture also requires a clear view of retention. If data is kept because it could support future inference or RAG, teams need to account for the cost of retaining it and the value of retrieving it. Retention decisions made only around the original task may not fit a system where outputs and records become inputs to later work.
Three questions for the next AI budget
Before expanding an AI program, finance and technology leaders can start with three practical questions:
- What is the retention policy for AI-generated content, and who set it?
- Does storage procurement reflect training-time performance requirements, or the full lifecycle of the data being produced?
- If AI usage doubled next year, would storage costs double as well, or grow faster?
Many organizations may not yet have confident answers. That leaves a concrete task for the next budget cycle: measure the data each workload creates, decide how long it needs to persist, and plan storage capacity around its likely reuse. As inference expands, the cost of every answer will include the data trail it leaves behind.
This content was produced by WD. It was not written by MIT Technology Review’s editorial staff.