Storage's role in generative AI

Storage's role in generative AI

Generative AI can only produce a good outcome after being trained on reams of data, according to Khalid Eidoo, co-founder and CTO of Crater Labs, an AI and machine learning company based in Toronto that works with businesses to solve specific problems using AI. One method Crater employs is a type of generative AI called generative adversarial networks (GANs), which it used to identify potential structural defects in welds when constructing a nuclear power plant.

In this case, the GAN, which uses four different neural networks, produces images that then get reconciled. Out of the hundreds of thousands of images generated, only five or six meet the high quality level needed, Eidoo said.

To support this functionality, Crater needed high-throughput storage that could read and write synchronously and chose Pure Storage's FlashBlade product. "When dealing with generative networks, you're simultaneously reading millions of images to write millions of images," Eidoo said.

GPUs play an important role in generative AI by accelerating the training of models. But when working with millions of images, the GPU buffer quickly fills up and images need to be written quickly to storage, Eidoo said. High-throughput storage can reduce the potential for a data bottleneck.

Flash not necessary, but optimal

High IOPS storage can provide a user experience more like high-performance computing, according to Matchett.

"You can do parallel file systems on a large number of spinning disks in aggregate," Matchett said.

A parallel file system feeds data from the LLMs to the GPUs, like DDN's A3I that combines DDN's Exascalar, parallel file system with NVIDIA's DGX, Matchett said.

A hybrid version of Exascalar could be used for generative AI, but it caches and tiers storage, potentially affecting performance, McDowell said. The GPUs can't sit idle, so the aggregated HDD performance will be cached to SSDs that operate faster than memory.

"[Those] that are serious about large language models, they're buying high-end flash storage," McDowell said.

Flash provides high IOPS in denser footprints and can also provide LLMs with aggregated performance, Eidoo said. It's possible to use millions of HDDs, but footprint matters. Flash storage is denser, higher performing and uses less power than HDDs. Technology that reduces power consumption now will benefit generative AI in the future.

"GPUs use power like there's no tomorrow," Eidoo said.

Cloud vs. on premises

LLMs also need space to train models. Whether that is on premises, in the public cloud or a hybrid of the two depends on the size of the model and the performance and control needed, Matchett said.

If generative AI is used for research, storing LLMs on the cloud is ideal because users can get the scale required without investing in the capex infrastructure. However, Matchett predicts vendors will offer generative AI applications that will become core to their business platforms. For those that are dependent on performance and security, on-premises storage will be key.

"As an enterprise operation, you've got production workloads that are running at some level of continuity, and that can get expensive," Matchett said.

Before choosing Pure Storage, Crater Labs worked with AWS and Google Cloud before moving to a hybrid infrastructure for speed, security and costs. Crater considered NetApp and HPE before choosing Pure.

Now, Crater Labs uses a combination of on premises -- FlashBlade and FlashBlade's built-in connection to an S3 object store bucket, according to Eidoo. Crater generates terabytes of data per week, which is inefficient to store solely on premises. Using the S3 object store lets Crater access images on the cloud for modeling.

"We knew very quickly as we started developing these generative models that the performance we were getting in the cloud wasn't adequate," Eidoo said.

Adam Armstrong is a TechTarget Editorial news writer covering file and block storage hardware and private clouds. He previously worked at StorageReview.com.