- GenAI can improve productivity by up to 66%.
- Most workers bring their own GenAI tools to work, whether their managers know it or not.
- As AI becomes more advanced, organizations are leveraging strategies to manage security risks and rising costs.
Workers around the world are reaping the benefits of AI to improve workflows and drive innovation. But as the widespread use of generative AI skyrockets across teams and functions, organizations must ensure it is deployed securely and cost-effectively.
According to recent research from May 2024, 75% of knowledge workers globally are already using GenAI applications, up 46% from six months earlier. And employees aren’t waiting for management guidance to get started: more than 78% are bringing their own GenAI tools to work.
It’s easy to understand why employees are so eager to use GenAI: it can improve productivity by up to 66%. But increased use of GenAI also comes with increased risk, particularly for organizations without a formal AI strategy. A recent study found that 71% of organizations provide no guidance to employees on when, where, and how to use AI.
“Allowing everyone to incorporate GenAI applications and use them however they want will create serious security and governance challenges,” said Fuzz Hussain, senior marketing manager for the AI portfolio at Dell Technologies. “Organizations should remember the hard lessons they learned when cloud computing was introduced and lax controls led to shadow IT. If they are not careful now, they will end up with shadow AI.”
As with shadow IT, shadow AI will create spiraling costs and data silos, as well as security risks.
The good news is that organizations can avoid past mistakes by creating an AI strategy for the future. By making the right strategic decisions about development and deployment, they can maximize control while also optimizing cost efficiency.
Solutions for GenAI implementation
Companies have several options for creating and deploying GenAI, and should carefully weigh the security and cost factors involved. Below are some of your options:
- Building a Large Language Model (LLM) from Scratch
Developing an LLM from scratch requires immense resources and specialized expertise, making it prohibitively expensive for most organizations.
- How to access an LLM through an application programming interface (API)
API-based LLMs often require data processing by third parties, which introduces security risks and places them out of reach of those using PPI or other information subject to privacy or compliance regulations.
Fees are based in part on the number of queries users submit, which can make applications difficult to evaluate or scale. And that’s not the only expense.
“You’re also paying for the hardware, software and maintenance of the solution – those costs are included in the price,” Hussain said.
- Building applications on public cloud infrastructure
Building in the public cloud allows developers to fine-tune models more than they can with APIs. They can also manage costs, but only to a certain extent.
“Many companies believe they are hitting the ‘easy’ button by subscribing to the amount of computing they think they will need. But the costs can add up quickly,” Hussain said.
Estimating compute resources, especially for a new capability like GenAI, can be difficult. Companies must also consider the costs of storing and moving data.
“Tuning a model and moving data, especially to different regions, involves costs,” Hussain added. “And as with API services, you pay for the use of the vendor’s hardware and software.”
- Download and customize an open source LLM
After downloading an open source model like Mistral or Meta Llama 3, you can adjust it to your needs or provide the model with your company’s data to improve its responses through a process called retrieval augmented generation (RAG).
RAG instructs the LLM to retrieve authoritative information relevant to your use cases. You can input your own data on-premises, without needing to send it outside the company.
For organizations looking to embark on their GenAI implementation journey, Hussain says RAG is a great place to start.
“You avoid having to train a model from scratch and you can leverage the data internally, keeping it protected wherever it goes,” he said.
Once a solution has been developed, employees can use it anywhere, performing inferences across on-premises data centers, in the cloud, or even on their own computers. Data is always protected by the company’s own security and governance protocols. And since applications are built and deployed in-house, there are no service fees.
Proven profitability
By keeping company data in-house, a custom open source LLM offers a clear security advantage. It also offers better cost control. But what difference does that make?
Tech Target’s business strategy group conducted an economic analysis in collaboration with Dell to find out. Researchers found that running GenAI inference on enterprise infrastructure is up to 75% more cost-effective than using the public cloud and up to 88% more cost-effective than using an API service.
A similar economic analysis conducted by Principled Technologies in collaboration with Dell found that running GenAI inference and fine-tuning solutions with Dell Technologies on-premises can be up to 74% more cost-effective than the public cloud.
The benefits are multiplied for organizations with thousands of users and larger LLMs, which can take advantage of economies of scale.
Moving forward with GenAI
Regardless of size, companies shouldn’t wait to get started with GenAI, Hussain said.
“Employees are already using it, so if you don’t have a plan in place, you’re essentially succumbing to shadow AI,” he said. “There are a variety of architectures you can use to safely and cost-effectively integrate GenAI into your ecosystem, and begin to unleash its power for your business.”
Click here to learn more about how Dell solutions can help you deploy GenAI securely and cost-effectively.
This post was created by Insider Studies with Dell.
