Cloud computing and generative AI are transforming the capabilities of data-driven enterprises by offering powerful analytics and business insights. Cloud services provide the essential infrastructure and tools for developing and deploying generative AI technologies. However, this progress comes with its own set of challenges, notably the surge in data volumes and escalating cloud infrastructure costs.
A recent 2024 State of Big Data Analytics report by SQream, a GPU-based big data platform, highlights the financial strain imposed by cloud analytics costs on data-driven enterprises. The study, which surveyed 300 senior data management professionals from U.S. companies, found that 71% frequently encounter unexpected high cloud analytics charges. Specifically, 5% of companies experience cloud “bill shock” monthly, 25% every two months, and 41% quarterly.
Despite substantial budgets, a staggering 98% of companies faced machine learning (ML) project failures in 2023 due to soaring cloud costs. Bill shocks occur when data workflows become too complex or large for the existing cloud query engine, leading to higher compute power requirements and associated costs.
“As data and analytics advance, companies are forced to limit dataset size and reduce complexity to manage expenses, impacting the quality of their insights,” said Deborah Leff, Chief Revenue Officer at SQream. “Many AI/ML projects are not initiated due to the high cost of experimentation over the cloud. Poor data preparation and inadequate data cleansing methods are other major contributors to project failures.”
The cost of running data and AI technologies over the cloud has been a significant deterrent. Cloud cost inflation is set to persist in 2024, necessitating cost-cutting measures within enterprises. Data from the Bureau of Labor Statistics’ Producer Price Index (PPI) for 2024 shows a month-over-month increase in data processing and related services, a category that includes cloud computing, with a current year-over-year uptick standing at 3.7%.
Nearly half of the enterprises (48%) in the SQream study admitted to reducing the complexity of queries to manage analytics costs, particularly concerning cloud resources and compute loads. Meanwhile, 46% are limiting AI-powered projects due to cost constraints.
Beyond vendor pricing, businesses often do not thoroughly analyze which in-house IT assets would benefit from cloud migration. “Cost is a major factor in project failures because expenses often escalate during experimentation. It’s not that machine learning architecture fails, rather management chooses to halt investment when costs spiral. Time to value is crucial, and experimenting often leads to high costs due to the size and complexity of modern data,” added Leff.
A third of the companies surveyed (33%) said they are using 5-10 solutions/platforms for data preparation, making this task extremely complicated. Using different tools by several users in parallel can create bottlenecks and hinder process analysis.
“The data center ecosystem, built on 40-year-old technology, needs modernization. Sticking with outdated methods is not the solution. Instead, companies should explore innovative approaches to avoid letting costs and data limitations restrict their analytics capabilities,” Leff said. “Tools like NVIDIA Rapids are valuable but require developer skills, highlighting the need for more accessible solutions. Companies must challenge the status quo and seek better options to overcome current constraints.”
As companies navigate market disruptions caused by generative AI and the rise of large language models (LLMs), the explosion in data volume and complexity makes ML technologies essential for market competitiveness. Limiting data queries for AI systems to manage costs results in superficial insights, leading to premature project termination. Ninety-two percent of companies in the study said they are actively working to “rightsize” their cloud spending on analytics to better align with their budgets.
Leff explained that GPU acceleration, despite perceptions of high expense, can significantly reduce costs while speeding up processing. This solution provides the benefits of the cloud with right-sized parallel processing resources and a flexible pay-as-you-go pricing option for agility and simplified management. Enterprises can rent the GPU resources they need and later automatically scale on-demand.
“NCBA, a large online bank with up to 60 million daily users, initially took 37 hours to update their marketing models with daily click data. Despite optimizing their queries and exploring expensive hardware solutions, this delay left them unable to use data strategically. When they turned to GPU acceleration, it helped reduce their data pipeline cycle time to just seven hours, enabling them to update models rapidly each day,” she added.
Leff emphasized that companies must think proactively and push the boundaries of what’s possible. The rapid evolution of generative AI highlights that current data strategies may not be sufficient. She predicted that the next two years would bring dramatic changes within the IT sector.
“We must envision and prepare for a future where data grows and queries become more complex, but outdated limitations are removed. Embracing new methods such as GPU acceleration can unlock significant value, and those who act quickly will reap the rewards,” she said.
