Building Customer-Facing AI Agents in 2025: From the "How" to the "What" and Back Again
TL;DR: We used to have to specify every single step in an AI agent's conversation flow, which was rigid, inflexible, and a lot of work. Then LLMs came along, promising to revolutionize the way we build AI agents by allowing us to generate adaptive responses on the fly, while only needing us to provide some general background context for their operation. But now, in 2025, we're realizing that LLMs do a poor job at filling in the gaps toward an effective customer interaction, and that the best way to build customer-facing AI agents is to combine the power of LLMs with traditional conversation design techniques.
A Bit of Generative AI History
Back in early 2023, when GPT 4 just came out, I—along with my co-founder Dor Zohar, and thousands of other developers—were excited about the new possibilities of building customer-facing AI agents. These "new agents" could handle complex interactions and use cases, creating a new and better kind of user experience.
But what we (like many others new to NLP) were totally unaware of at the time was that enterprises had already been building complex AI agents for years, using pretty much the same Transformer models invented in 2017. These had roughly the same architecture that powers today's state-of-the-art LLMs. They were just smaller and more limited in functionality and reliability.
The way those traditional agents were built was by defining a flowchart of the conversation, and then writing utterances for each node in the flowchart. The utterances were written by conversation designers, who were trained to write in a way that was compliant with brand voice and legal requirements, and to make the AI engaging and effective and resolving customer requests.
The transition from one node to another was performed using NLP classifiers, which would classify the user's input into one of the predefined intents. The intent would then determine the next node in the flowchart. These classifiers were often based on those early transformer models, which were much smaller and less capable than today's LLMs.
The Limitations of Traditional AI Agents
While this approach worked well for many years, it had its limitations. The flowcharts were rigid and inflexible, and the utterances were often too generic or too specific. The classifiers were also limited in their ability to understand the nuances of natural language, which led to misclassifications and poor user experiences.
It also meant that the agents were not very adaptive. Users were confined to pre-defined steps and couldn't express themselves freely like in a normal conversation, and were often frustrated by the experience, as the agents were unable to handle unexpected or mistimed inputs or changes in context. This rigidity made it difficult to create engaging and effective AI agents.
Consequently, user adoption was low, and many companies struggled to get their customers to use these agents, as many users preferred to speak with a human agent instead—completely defeating the purpose of building an AI agent in the first place.
Not only did these hard limitations exist, but, worse, the effort in building an maintaining such agents was nonetheless quite considerable, and only the largest companies were normally able to justify the investment. Conversation designers had to write and maintain thousands of utterances, and the flowcharts (and corresponding NLU classifier models) had to be updated whenever a new intent was added or an existing one was modified. This made the process slow and cumbersome, and it often resulted in outdated or inaccurate agents.
But, despite all of that, these traditional AI agents were (and still are) superior to LLMs in many ways. They were more reliable, more predictable, and more compliant with brand voice and legal requirements. They also provided a better user experience in the so-called "happy-paths" of customer interactions, as they were designed to handle specific use cases and interactions with maximum efficiency and grace, driven by the deep domain understanding of their designers.
Get in touch with us
The New Silver-Bullet: Forget the "How", Just Define the "What"
Then came the LLMs. They promised to revolutionize the way we build AI agents by allowing us to generate responses on the fly, without the need for predefined designs or flowcharts. This was a game-changer, and many companies jumped on the bandwagon, eager to leverage the power of LLMs.
The idea was that if you provide the LLM with some general background context about the agent's purpose and the domain it operates in, it would be able to generate appropriate responses to user inputs, without the need for predefined utterances or flowcharts. This meant that you could build AI agents that were more adaptive, more flexible, and more engaging than ever before—at a fraction of the effort.
Heck, you didn't even need to know what you were doing! You could just throw at the LLM some auto-parsed background material from your company's documentation, and it would generate a response that was "good enough" for most use cases. This seemed like a huge leap forward, and it opened up new possibilities for building AI agents that were more human-like and more engaging.
And it seemed like anyone could build them. It seemed like Conversational AI was finally accessible to everyone, not just the big enterprises with full-time conversation designers and ML engineers on staff.
No Silver Bullet After All
But, as we all know, the reality was not quite as rosy as the promise. LLMs were great at generating text, but they were not very good at understanding the nuances of customer interactions. They often produced responses that were irrelevant, inaccurate, or even offensive. They also struggled with maintaining context and coherence over long conversations, which led to frustrating user experiences.
Beyond that, these new LLM agents lacked an important, fundamental quality: a sense of competence and design.
Instead of giving customers a feeling of understanding, assurance, and helpfulness, they were passive and reactive, often responding to user inputs in a way that felt disjointed and unplanned.
They were not designed to handle specific use cases or interactions efficiently, and they often produced responses that made customers feel as if they're talking to a disoriented, compulsively people-pleasing service rep on their first day on the job.
Get in touch with us
The Return of the "How"
There's a hidden lesson here—which seems to have taken a lot of trial and error to uncover.
Just as we wouldn't trust a complete stranger to fill in the blanks in a sensitive customer interaction, we can't expect LLMs to always infer our intentions, preferences, or requirements.
Left to their own devices, LLMs will need to make guesses—but their guesses may often misalign with our brand, our service standards, or even basic user expectations. Sometimes, they'll get it wrong in subtle ways; other times, the results can be blatantly off-mark or even harmful to the business.
This is why the reality isn't as simple as "Let the LLM figure out what to do." While LLMs excel at generating plausible responses, they lack the context, priorities, and guardrails that come from explicit design and domain expertise.
We need to provide a system of guidance—a framework that tells the LLM not just what the goal is, but how we want to achieve it. This means specifying, at least for critical parts of the interaction, the steps, tone, and boundary conditions that matter most.
This is exactly what we at Parlant have been working on, and it's why we're so excited about the future of AI agents in 2025.
Moving away from the rigidity of traditional flowcharts was necessary, but swinging to the opposite extreme—where we only define the "what" and leave the "how" entirely up to the model—creates new problems.
LLMs need us to step in and explain, sometimes in detail, how we want certain conversations to unfold. And like I said, this isn't just about delivering a better user experience; in many industries, especially those with strict service protocols or regulatory requirements, it's essential for compliance and risk management. It's a prerequisite for safe production deployment.
Ultimately, the most effective AI agents in 2025 are those that blend the adaptability of LLMs with the intentionality and structure of conversation design. By guiding LLMs with clear instructions on both the "what" and the "how," we can create agents that are not only flexible and engaging, but also reliable, compliant, and truly aligned with our goals.
Get in touch with us