Private AI could drive demand for edge compute

By J. Sharpe Smith, Contributing Editor

 

Only a year ago, artificial intelligence must have seemed out of reach for many enterprises, because it required expensive GPUs running in the public cloud. Chipmakers have been responding with new solutions targeting Private AI.

 

Intel recently launched a Private AI Collaborative Institute and has said its latest enterprise AI offering, introduced last April, will address the challenges businesses face in scaling AI initiatives, because only 10% of enterprises had moved GenAI projects into production last year.

 

Broadcom is also hearing from customers that public cloud is not the only way to leverage AI.  In conversations with the heads of AI at nearly 200 end-user organizations, it became clear to Chris Wolf, Global Head of AI and Advanced Services at Broadcom's VMware Cloud Foundation Division, that organizations will leverage both public clouds and private data centers (owned or leased capacity) to meet their needs.

 

“Customers with mature AI environments have shared with me that their cost savings for Private AI is three to five times that of comparable public cloud AI services,” Wolf wrote recently on the company’s website.

 

Private AI refers to an AI environment built by or for a specific organization, to be used exclusively by that organization, according to Equinix. As one of the largest operators of data centers that lease capacity to enterprises, Equinix should see even more demand for its services as enterprises train and apply Private AI models.

 

Enterprises have always wanted to keep their data secure and segregated from outside data, but some have found that this is not always possible when using public LLMs. So cost is not the only reason companies are considering their own AI initiatives.

 

“Many companies don't want their data going up to a public cloud somewhere, because every large network has been hacked at this point in time. There's ransomware out there, and there's a lot of bad guys that are getting into your data,” according to Joe Madden, Principal Analyst at Mobile Experts. “The best way for an enterprise to control and safeguard data is to keep it inside their building, control the passwords and restrict access to the data,” he added.

 

Bring on Edge Compute

 

The growth of Private AI will increase the need for low latency in data analysis in the enterprise, thus creating another potential driver for edge computing. “Edge has a role to play in the evolution of AI and generative AI for businesses. It enables data analysis in real- or near-real time, making training AI models a simpler task and improving the performance of AI-driven applications,” Accenture said in a research report.

 

“Since AI is deployed to make immediate critical decisions such as quality inspection, surveillance and alarm management, any latency within the system may result in machine stoppage or slow down causing heavy damages or loss in productivity. Moving AI to the edge mitigates potential vulnerability and risks such as unreliable connectivity and delayed responses,” explains Lian Jye Su, Principal Analyst at ABI Research.

 

Fortune Business Insights projects growth from $15.96 billion in 2023 to $216.76 billion by 2032, a CAGR of 33.6%.

 

The edge computing market is an ecosystem that includes cloud giants such as Amazon Web Services (AWS) and Microsoft; infrastructure providers such as Advanced Micro Devices and Ericsson; and edge-as-a-service providers such as TATA Communications and Orange Business Services. AWS and Microsoft, which are known for regional, public data centers, are the two leading companies in developing edge computing capabilities in commercial products, according to Madden. For example, Amazon provides the AWS Outpost, which is a scaled down version of its hyperscale data center, in a rack that the enterprise can put into a factory or other facility and run the AWS stack with their own private data.

 

Edge Computing Ecosystem Still Developing

 

Mobile Experts has been studying edge computing for more than five years and believes that it is essential to the growth of private cellular. An edge compute server can be used for many purposes in an enterprise, including processing data from IoT devices and compressing and analyzing huge amounts of video data to fulfill the vision of Industry 4.0 in various vertical markets.

 

IoT devices are expected to grow from 16.1 billion to 39.9 billion in 2033, a CAGR of 10%, according to Transforma Insights. More than 2.5 billion small, low-power devices are projected to ship in 2030 for use in embedded machine learning, known as TinyML, which will also add to data usage, according to ABI Research.

 

“The Fourth Industrial Revolution, or Industry 4.0, requires multiple components, including IoT devices, spectrum, a radio network, as well as an edge computing server. And, most important, there has to be a business model that makes it all work,”  Madden said. “Failure in any one of those things can kill the project and make it non viable.”

 

To describe the Industrial Private Cellular (INDPRIVATE) ecosystem, Madden uses a wheel that depends on a number of spokes, including the core network, the RAN, spectrum availability, devices, a clear business case, aligned business model, distribution channel, Information technology and operational technology (IT/OT) applications, edge compute, and integrated network management.

 

“For the Industrial Private Cellular market to be successful, each of the elements shown in the diagram should be well developed,” Madden said. “If the diagram looks like a wheel, the market can ‘roll’ forward.” One look at Madden’s graphic shows a private cellular market that is definitely not ready to speed forward.

 

To realize its potential, the ecosystem for industrial cellular needs the most development in the areas of devices and integrated network management, according to Mobile Experts. 

 

“For the whole thing to work together, there needs to be hundreds of system integrators that pop up with expertise in specific verticals, in specific applications,” Madden said. “Maybe some are going to specialize in drones, and other ones will specialize in cars, and maybe others will specialize in manufacturing.”

 

Mobile Experts has tracked the exponential growth of edge data centers from less than 500 in 2019 up to more than 6,000 in 2024. Madden sees a long runway of continued growth in the future.

 

“There is going to be 30 years of growth in edge computing and private wireless customization,” he said. “There's a lot that can be done. Almost every business on the planet has something they want to automate.”