Large language models (LLMs) have grabbed the world’s attention for their seemingly magical ability to instantaneously sift through endless data, generate responses, and even create visual content from simple prompts. But their “small” counterparts aren’t far behind. And as questions swirl about whether AI can actually generate meaningful returns (ROI), organizations should take notice. Because, as it turns out, small language models (SLMs), which use far fewer parameters, compute resources, and energy than large language models to perform specific tasks, have been shown to be just as effective as their much larger counterparts.
In a world where companies have invested ungodly amounts of money on AI and questioned the returns, SLMs are proving to be an ROI savior. Ultimately, SLM-enabled agentic AI delivers the best of both SLMs and LLMs together — including higher employee satisfaction and retention, improved productivity, and lower costs. And given a report from Gartner that said over 40% of agentic AI projects will be cancelled by the end of 2027 due to complexities and rapid evolutions that often lead enterprises down the wrong path, SLMs can be an important tool in any CIO’s chest.
Take information technology (IT) and human resources (HR) functions for example. In IT, SLMs can drive autonomous and accurate resolutions, workflow orchestration, and knowledge access. And for HR, they’re enabling personalized employee support, streamlining onboarding, and handling routine inquiries with privacy and precision. In both cases, SLMs are enabling users to “chat” with complex enterprise systems the same way they would a human representative.
Given a well-trained SLM, users can simply write a Slack or Microsoft Teams message to the AI agent (“I can’t connect to my VPN,” or “I need to refresh my laptop,” or “I need proof of employment for a mortgage application”), and the agent will automatically resolve the issue. What’s more, the responses will be personalized based on user profiles and behaviors and the support will be proactive and anticipatory of when issues might occur.
Understanding SLMs
So, what exactly is an SLM? It’s a relatively ill-defined term, but generally it is a language model with somewhere between one billion and 40 billion parameters, versus 70 billion to hundreds of billions for LLMs. They can also exist as a form of open source where you have access to their weights, biases, and training code.
There are also SLMs that are “open-weight” only, meaning you get access to model weights with restrictions. This is important because a key benefit with SLMs is the ability to fine-tune or customize the model so you can ground it in the nuance of a particular domain. For example, you can use internal chats, support tickets, and Slack messages to create a system for answering customer questions. The fine-tuning process helps to increase the accuracy and relevance of the responses.
Agentic AI will leverage SLMs and LLMs
It’s understandable to want to use state-of-the-art models for agentic AI. Consider that the latest frontier models score highly on math, software development and medical reasoning, just to name a few categories. Yet the question every CIO should be asking: do we really need that much firepower in our organization? For many enterprise use cases, the answer is no.
And even though they are small, don’t underestimate them. Their small size means they have lower latency, which is critical for real-time processing. SLMs can also operate on small form factors, like edge devices or other resource-constrained environments.
Another advantage with SLMs is that they are particularly effective with handling tasks like calling tools, API interactions, or routing. This is just what agentic AI was meant to do: carry out actions. Sophisticated LLMs, on the other hand, may be slower, engage in overly reasoned handling of tasks, and consume large amounts of tokens.
In IT and HR environments, the balance among speed, accuracy, and resource efficiency for both employees and IT or HR teams matters. For employees, agentic assistants built on SLMs provide fast, conversational help to solve problems faster. For IT and HR teams, SLMs reduce the burden of repetitive tasks by automating ticket handling, routing, and approvals, freeing staff to focus on higher-value strategic work. Furthermore, SLMs also can provide substantial cost savings as these models use relatively smaller levels of energy, memory, and compute power. Their efficiency can prove enormously beneficial when using cloud platforms.
Where SLMs fall short
Granted, SLMs are not silver bullets either. There are certainly cases where you need a sophisticated LLM, such as for highly complex multi-step processes. A hybrid architecture — where SLMs handle the majority of operational interactions and LLMs are reserved for advanced reasoning or escalations — allows IT and HR teams to optimize both performance and cost. For this, a system can leverage observability and evaluations to dynamically decide when to use an SLM or LLM. Or, if an SLM fails to get a good response, the next step could then be an LLM.
SLMs are emerging as the most practical approach to achieving ROI with agentic AI. By pairing SLMs with selective use of LLMs, organizations can create balanced, cost-effective architectures that scale across both IT and HR, delivering measurable results and a faster path to value. With SLMs, less is more.
—
New Tech Forum provides a venue for technology leaders—including vendors and other outside contributors—to explore and discuss emerging enterprise technology in unprecedented depth and breadth. The selection is subjective, based on our pick of the technologies we believe to be important and of greatest interest to InfoWorld readers. InfoWorld does not accept marketing collateral for publication and reserves the right to edit all contributed content. Send all inquiries to doug_dineley@foundryco.com.
