What changed
The article highlights that AI inference, which involves running pre-trained models to generate responses, is creating a substantial increase in data storage costs. Unlike training, which is a finite process, inference runs continuously and generates large amounts of back-end data.
For example, an 8-second AI-generated video clip, returned to a user as a 3.97MB file, leaves more than 500MB of associated data behind, including logs, cached context, immediate renders, backups, and safety records.
This represents a 130x increase over the output file size. As AI inference demand is projected to grow at a 35% CAGR through 2030, the data footprint is expected to expand rapidly, with storage costs outpacing compute budgets.
What this means in practice
For organizations using AI for tasks like generating product videos or processing claims documents with retrieval-augmented generation (RAG), storage costs are compounding with each task.
Unlike compute costs, which are often predictable and budgeted, storage costs grow with every video generated or claim processed, as the back-end data does not disappear after the task is completed.
This means that storage costs can quickly become a significant portion of an AI program's total cost, especially as usage scales.
The article emphasizes that storage must support not only growing capacity but also dynamic access to data, as more data is retained, retrieved, and reused.
Limitations and availability
The article does not specify exact storage solutions or technologies, but it notes that HDDs provide a cost-effective, high-capacity foundation for retaining large volumes of data.
The challenge lies in designing storage infrastructure that balances performance with capacity, power, reliability, and total cost of ownership across the full AI data life cycle.
The article also highlights that most organizations lack confidence in answering key questions about their AI infrastructure budgets, such as data retention policies and storage procurement strategies. This gap needs to be addressed before the next budget cycle.
