6 posts tagged with "conversation-modeling"

Here's something not many know about our team. While we're now leading the development of Parlant—the open-source Conversational AI engine for reliable customer interactions—most of us didn't actually come from the NLP world.

Actually, our primary experience was in building mission-critical, real-time systems. Coming into NLP from that world was a bit of a shock, to be honest. We were used to ultimate control and determinism as a non-negotiable baseline, yet NLP offered very little of that; in fact, too little. And that was exactly our problem with it—and what we set out to solve.

The Three Challenges That Led to Parlant​

So, about a year ago, after getting quite frustrated with the available options for building reliable Conversational AI with LLMs, our team set out to build a better solution, what today is Parlant. After extensive research on technology and countless interviews with others building CAI for the past few years, we set out to solve three big challenges that we've identified:

1. Nobody had a good idea of how to get started with building scalable Conversational AI agents with LLMs. Everybody was reinventing the wheel, and most were following development patterns that were doomed to fail from the start, simply because they were accessible and easier to wrap your head around as a newcomer.

2. Agent complexity couldn't scale. Either prompts got so intractable that the LLM would consistently fail to produce the desired results, or, otherwise, you had to divide your agent's knowledge into different "graph nodes" with isolated prompts that were exclusively specialized for a particular topic, paving the way to the agent's too-frequent incoherence, misinterpretation of conversation history, and, consequently, generating misleading, non-aligned outputs to users.

3. Production agents required efficient collaboration between project stakeholders, particularly the business experts and the developers. However, translating changing business requirements into your agent was notoriously difficult. You couldn't even trust the changes you were making, which begged the question: "Why even bother specifying them?" This has frustrated many people with LLM projects in general, which has unfortunately led to many projects being abandoned after several months of development.

"It doesn't have to be like this" was the thought that set us on our path to building Parlant.

Get in touch with us

Our Core Insight: Contextual Guidelines​

Before diving into how we built Parlant's architecture, let me share the key innovation that makes it all work—because this was crucial to fixing the feedback cycle between business experts and developers.

Like I said before, one of the major issues in building customer-facing AI agents is the feedback cycle between business experts and developers. Business experts should be empowered to provide feedback clearly, and developers should be able to implement such feedback easily, quickly, and reliably.

It was clear to us that, as this feedback comes from business experts, it cannot be expected to come in the form of intricate flow-charts and things of that nature. Instead, it had to go through as few "artificial transformations" as possible, from the moment the business expert thinks about it to the moment it is turned into a requirement.

We then identified that when something doesn't work the way it should, a business expert looking at it can usually go, "Oh, that's not how it should be. Instead, it should be like that." But then the issue was that sometimes it should be like that, but other times it shouldn't, depending on the context.

For example, if the business expert says, "Oh, here the agent should be more proactive and push for a sale," it doesn't mean that the agent should act that way in all circumstances. It depends on the context.

This led to a key insight and design decision: Contextual Guidelines. Each guideline in Parlant would have a condition, specifying the set of circumstances in which it should be applied, and an action, which is the actual guidance for the agent's behavior.

This meant that business experts only had to do one thing whenever they desired a change in the agent's behavior: to specify that, "When [some condition holds], then [do something]." And this is extremely doable.

Of course, today, Parlant takes it even a step further. It supports things like guideline relationships, which effectively add another layer of control and precision to your agent's behavior. While not strictly needed in many cases, there are times when you really want to improve user engagement by a few extra percents, and Parlant's fine-grained control mechanisms will be there for you.

Get in touch with us

Parlant's Architectural Vision​

With our core innovation defined, we had to build an architecture that could support it reliably. The first thing we had to recognize was that LLMs were fundamentally inconsistent. This led to two critical design decisions:

1. We needed to try to simplify the tasks that LLMs had to do. This meant breaking down the conversational processing pipeline into multiple, manageable steps, each with its own single responsibility.

To take just one example, in Parlant, the engine first has to determine what guidelines are immediately relevant for its next response. It then filters out all of the irrelevant ones, and only then does it pass the relevant guidelines to the LLM, which is now focused on a small set of clear instructions.

This way, we could ensure that the LLM's output was more predictable and aligned with the agent's intended behavior.

2. We had to guide every piece of code we wrote by tests, essentially following TDD (Test Driven Development) to ensure that everything is accounted for and that we're able to keep track of the engine's performance and consistency.

Today, Parlant's built-in test suite has hundreds of tests that cover all of the engine's functionality, and we run them every time we make a change to the codebase.

This way, we can be sure that our changes don't break anything and that the engine continues to perform as expected—even as new LLM versions are released.

From our extensive experience in software development, we knew that if we followed these two principles, we would be taking our best chances at taming this seemingly untameable technology.

Get in touch with us

First Steps: Getting the Conversation API Right​

Importantly, one of the things we had to implement was a robust conversation API so that we could easily and quickly set up new conversational agents without reinventing the wheel every time.

But we wanted to make sure that this API was not just a simple wrapper around an LLM. Instead, we wanted it to be a full-fledged conversation engine that could handle all of the complexities of real-world conversations.

We thus built Parlant's conversational infrastructure focusing on the following features:

Built-In Conversation Store​

A built-in persistent store of conversations, filled with all of the raw data on each conversation—on both the agent and the customer—because we know that data would be extremely valuable in analyzing agent performance.

Native Conversational Status Updates​

Support for status updates, such as "Thinking," "Typing," etc. All seamlessly baked right into the API.

Event-Level Debugging with Correlation IDs​

Each event in the conversation is associated with a correlation ID associated with the HTTP request that triggered its creation. This was needed to streamline debugging, so you could ask the system, "Give me all of the logs that came from generating this one particular event in this conversation."

Asynchronous, Free-Flowing Conversation API​

An asynchronous API so that we could build a GUI where conversations didn't necessarily have to work like ping-pong, where the customer (or the agent) can only send one message at a time. That's just not how real conversations work. Real people often send multiple messages at a time to get their point across.

So we wanted our API to be future-compatible and enable conversations that weren't just fluent in their content, but also in their flow.

Backend-Initiated Agent Responses​

Agent conversations needed to be triggerable asynchronously from the backend as well—for example, initiating a conversation with the user if you find some issue in their account, or, alternatively, updating them on the result of a long-running background task using a callback function.

Imagine your agent booking a flight ticket for a customer. This normally takes about 30-60 seconds to get sorted, but your agent can't just get stuck! It should still be responsive to your customer in their conversation. So we designed for that, too.

From Mission-Critical Systems to Conversational AI​

Looking back, our journey from building deterministic, mission-critical systems to creating Parlant has been about bringing that same rigor and reliability to the world of conversational AI.

Parlant's whole existence stems from a desire to bring more control, predictability, and quality into generative AI agents. And now, a year later, we know that we had taken the right approach. Parlant agents are famously consistent in their instruction-following ability compared to those built from scratch with lower-level LLM frameworks.

The principles that guided us in our previous work—breaking down complex problems, testing everything, and empowering stakeholders—turned out to be exactly what the conversational AI space lacked.

We've gotten feedback from many people since, saying things like, "After a few months of banging my head against these LLMs, I realized this is the way it should be done... and then I found Parlant has already done it.".

By applying these principles to the inherently unpredictable world of LLMs, we've created something that finally makes reliable customer-facing AI agents achievable. And we're proud to offer it to the community as an open-source solution.

Try It Yourself!​

If you haven't tried it yet, I encourage you to go through the official tutorial and see for yourself how seamless it is to create scalable agents in Parlant. It's all about getting the architecture right. When you do that, things naturally fall into place. And that's exactly what we've achieved with Parlant's architecture.

Get in touch with us

TL;DR: We used to have to specify every single step in an AI agent's conversation flow, which was rigid, inflexible, and a lot of work. Then LLMs came along, promising to revolutionize the way we build AI agents by allowing us to generate adaptive responses on the fly, while only needing us to provide some general background context for their operation. But now, in 2025, we're realizing that LLMs do a poor job at filling in the gaps toward an effective customer interaction, and that the best way to build customer-facing AI agents is to combine the power of LLMs with traditional conversation design techniques.

A Bit of Generative AI History​

Back in early 2023, when GPT 4 just came out, I—along with my co-founder Dor Zohar, and thousands of other developers—were excited about the new possibilities of building customer-facing AI agents. These "new agents" could handle complex interactions and use cases, creating a new and better kind of user experience.

But what we (like many others new to NLP) were totally unaware of at the time was that enterprises had already been building complex AI agents for years, using pretty much the same Transformer models invented in 2017. These had roughly the same architecture that powers today's state-of-the-art LLMs. They were just smaller and more limited in functionality and reliability.

The way those traditional agents were built was by defining a flowchart of the conversation, and then writing utterances for each node in the flowchart. The utterances were written by conversation designers, who were trained to write in a way that was compliant with brand voice and legal requirements, and to make the AI engaging and effective and resolving customer requests.

The transition from one node to another was performed using NLP classifiers, which would classify the user's input into one of the predefined intents. The intent would then determine the next node in the flowchart. These classifiers were often based on those early transformer models, which were much smaller and less capable than today's LLMs.

The Limitations of Traditional AI Agents​

While this approach worked well for many years, it had its limitations. The flowcharts were rigid and inflexible, and the utterances were often too generic or too specific. The classifiers were also limited in their ability to understand the nuances of natural language, which led to misclassifications and poor user experiences.

It also meant that the agents were not very adaptive. Users were confined to pre-defined steps and couldn't express themselves freely like in a normal conversation, and were often frustrated by the experience, as the agents were unable to handle unexpected or mistimed inputs or changes in context. This rigidity made it difficult to create engaging and effective AI agents.

Consequently, user adoption was low, and many companies struggled to get their customers to use these agents, as many users preferred to speak with a human agent instead—completely defeating the purpose of building an AI agent in the first place.

Not only did these hard limitations exist, but, worse, the effort in building an maintaining such agents was nonetheless quite considerable, and only the largest companies were normally able to justify the investment. Conversation designers had to write and maintain thousands of utterances, and the flowcharts (and corresponding NLU classifier models) had to be updated whenever a new intent was added or an existing one was modified. This made the process slow and cumbersome, and it often resulted in outdated or inaccurate agents.

But, despite all of that, these traditional AI agents were (and still are) superior to LLMs in many ways. They were more reliable, more predictable, and more compliant with brand voice and legal requirements. They also provided a better user experience in the so-called "happy-paths" of customer interactions, as they were designed to handle specific use cases and interactions with maximum efficiency and grace, driven by the deep domain understanding of their designers.

Get in touch with us

The New Silver-Bullet: Forget the "How", Just Define the "What"​

Then came the LLMs. They promised to revolutionize the way we build AI agents by allowing us to generate responses on the fly, without the need for predefined designs or flowcharts. This was a game-changer, and many companies jumped on the bandwagon, eager to leverage the power of LLMs.

The idea was that if you provide the LLM with some general background context about the agent's purpose and the domain it operates in, it would be able to generate appropriate responses to user inputs, without the need for predefined utterances or flowcharts. This meant that you could build AI agents that were more adaptive, more flexible, and more engaging than ever before—at a fraction of the effort.

Heck, you didn't even need to know what you were doing! You could just throw at the LLM some auto-parsed background material from your company's documentation, and it would generate a response that was "good enough" for most use cases. This seemed like a huge leap forward, and it opened up new possibilities for building AI agents that were more human-like and more engaging.

And it seemed like anyone could build them. It seemed like Conversational AI was finally accessible to everyone, not just the big enterprises with full-time conversation designers and ML engineers on staff.

No Silver Bullet After All​

But, as we all know, the reality was not quite as rosy as the promise. LLMs were great at generating text, but they were not very good at understanding the nuances of customer interactions. They often produced responses that were irrelevant, inaccurate, or even offensive. They also struggled with maintaining context and coherence over long conversations, which led to frustrating user experiences.

Beyond that, these new LLM agents lacked an important, fundamental quality: a sense of competence and design.

Instead of giving customers a feeling of understanding, assurance, and helpfulness, they were passive and reactive, often responding to user inputs in a way that felt disjointed and unplanned.

They were not designed to handle specific use cases or interactions efficiently, and they often produced responses that made customers feel as if they're talking to a disoriented, compulsively people-pleasing service rep on their first day on the job.

Get in touch with us

The Return of the "How"​

There's a hidden lesson here—which seems to have taken a lot of trial and error to uncover.

Just as we wouldn't trust a complete stranger to fill in the blanks in a sensitive customer interaction, we can't expect LLMs to always infer our intentions, preferences, or requirements.

Left to their own devices, LLMs will need to make guesses—but their guesses may often misalign with our brand, our service standards, or even basic user expectations. Sometimes, they'll get it wrong in subtle ways; other times, the results can be blatantly off-mark or even harmful to the business.

This is why the reality isn't as simple as "Let the LLM figure out what to do." While LLMs excel at generating plausible responses, they lack the context, priorities, and guardrails that come from explicit design and domain expertise.

We need to provide a system of guidance—a framework that tells the LLM not just what the goal is, but how we want to achieve it. This means specifying, at least for critical parts of the interaction, the steps, tone, and boundary conditions that matter most.

This is exactly what we at Parlant have been working on, and it's why we're so excited about the future of AI agents in 2025.

Moving away from the rigidity of traditional flowcharts was necessary, but swinging to the opposite extreme—where we only define the "what" and leave the "how" entirely up to the model—creates new problems.

LLMs need us to step in and explain, sometimes in detail, how we want certain conversations to unfold. And like I said, this isn't just about delivering a better user experience; in many industries, especially those with strict service protocols or regulatory requirements, it's essential for compliance and risk management. It's a prerequisite for safe production deployment.

Ultimately, the most effective AI agents in 2025 are those that blend the adaptability of LLMs with the intentionality and structure of conversation design. By guiding LLMs with clear instructions on both the "what" and the "how," we can create agents that are not only flexible and engaging, but also reliable, compliant, and truly aligned with our goals.

Get in touch with us

Or: Why “Can we turn off generation” might be the smartest question in generative AI

Not long ago, I found myself in a meeting with technical leaders from a large enterprise. We were discussing Parlant as a solution for building fluent yet tightly controlled conversational agents. The conversation was going well—until someone asked a question that completely caught me off guard:

“Can we use Parlant while turning off the generation part?”

At first, I honestly thought it was a misunderstanding. A generative AI agent… without the generation? It sounded paradoxical.

But I paused. And the more I considered it, the more the question started to make sense.

The High Stakes of Customer-Facing AI​

These teams weren’t playing around with demos. Their AI agents were destined for production—interfacing directly with millions of users per month. In that kind of environment, even a 0.01% error rate isn’t acceptable. One in ten thousand bad interactions is one too many when the outcome could be compliance failures, legal risk, or brand damage.

At this scale, “pretty good” isn’t good enough. And while LLMs have come a long way, their free-form generation still introduces uncertainty—hallucinations, unintended tone, and factual drift.

So no, the question wasn’t absurd. It was actually pivotal.

A Shift in Perspective​

Later that night, I kept thinking about it. The question made more sense than I had initially realized, because these organizations weren’t lacking resources or expertise.

In fact, they had full-time Conversation Designers on staff. These are professionals trained in designing agentic behaviors, crafting interactions, and writing responses that align perfectly with brand voice and legal requirements, and get customers to actually engage with the AI — which turns out to be no easy task in practice!

So they weren’t asking to turn off generation out of fear—they were asking to turn it off because they wanted—and were able—to take control into their own hands.

That’s when it hit me: we’ve been misframing what “generative AI agents” actually are.

They’re not necessarily about open-ended token-by-token generation. They’re about being adaptive: responding to inputs in context, with intelligence. Whether those responses come directly, token-by-token, from an LLM, or from a curated response bank, doesn’t actually matter. What matters is whether they’re appropriate: compliant, contextual, clear, and useful.

Get in touch with us

The Hidden Key to the Hallucination Problem​

Everyone is looking for a fix to hallucinations. Here’s a radical thought: we think it’s already here.

Conversation Designers.

Having conversation designers on your team—as many enterprises already do—you're not just mitigating output hallucinations, you're actually primed to eliminate them completely.

They also bring clarity into the customer interaction. Intentionality. An engaging voice. And, they create more effective interactions than foundation LLMs can, because LLMs (on their own) still don’t sound quite right in customer-facing scenarios.

So instead of trying to retrofit generative systems with band-aids, I realized: Why not bake this into Parlant from the ground up? After all, Parlant is all about design authority and control. It’s about giving the right people the tools to shape how AI behaves in the world. This was a perfect match—especially for these enterprise use cases which had so much to gain from adaptive conversations, if only they could trust them with real customers.

From Insight to Product: Utterance Matching​

That was the breakthrough moment that led us to build Utterance Templates into Parlant.

Utterance Templates let designers provide fluid, context-aware templates for agent responses: responses that feel natural but are fully vetted, versioned, and governed. It’s a structured way to maintain LLM-like adaptability while keeping a grip on what’s actually said.

Under the hood, utterances templates work in a 3-stage process:

1. The agent drafts a fluid message based on the current situational awareness (interaction, guidelines, tool results, etc.)
2. Based on the draft message, it matches the closest utterance template found in your utterance store
3. The engine renders the matched utterance template (which is in Jinja2 format), using tool-provided variable substitutions where applicable

We immediately knew this would work perfectly with Parlant's hybrid model: one that gives software developers the tools to build reliable agents, while letting business and interaction experts define how those agents behave. And the guys at that particular enterprise immediately knew it would work, too.

Conclusion: Empower the Right People​

The future of conversational AI isn’t about removing people from the loop. It’s about empowering the right people to shape and continuously improve what AI says and how it says it.

With Parlant, the answer can be: the people who know your brand, your customers, and your responsibilities best.

And so that’s not absurd at all. It’s exactly how it should be. At least in our view!

Get in touch with us

We're excited to announce Parlant 2.2, and with it, one of our most requested features to date: a fully capable Python SDK.

Until now, building Parlant agents meant working exclusively through the CLI. While powerful, that approach had limits — especially for developer-first teams wanting to programmatically compose complex agent behavior.

That changes today.

Get in touch with us

💡 Why a Python SDK?​

At its core, Parlant is about precision control over conversational AI. We believe that production-grade agents can’t rely on LLM guesswork. You need to define conditions, enforce behavior, and own the outputs — not hope for the best.

The new SDK lets you do exactly that, entirely in code.

You can now spin up agents, define condition-based journeys, wire up tools, connect utterance templates, and even express relationships between guidelines — all without touching the CLI.

🧠 Build Smarter Agents — Programmatically​

Here’s a taste of what you can now do with just a few lines of Python:

import parlant.sdk as p
from textwrap import dedent
import asyncio

@p.tool
async def human_handoff_to_sales(context: p.ToolContext) -> p.ToolResult:
  await notify_sales(context.customer_id, context.session_id)

  return p.ToolResult(
    data="Session handed off to sales team",
    control={"mode": "manual"},
  )

async def main() -> None:
    async with p.Server() as server:
        agent = await server.create_agent(
            name="Otto Carmen",
            description="You work at a car dealership",
        )

        journey = await agent.create_journey(
            title="Sell a car",
            triggers=["The customer wants to buy a new car"],
            description=dedent("""\
            Proactively help the customer decide what new car to get.

            1. First vibe and seek to clarify their situation and needs.
            Chit-chat about their current car, what they like or don't,
            as well as whether they have children or pets.
            2. Ask them about budget and preferences.
            3. Once needs are clarified, recommend relevant categories
            or specific models for consideration.
            4. Once a choice is made, ask them to confirm the order"""),
        )

        transfer_to_human_sales_rep = await journey.create_guideline(
            condition="the customer clearly stated they wish to buy a specific car",
            action="transfer them to the sales team",
            tools=[human_handoff_to_sales],
        )

asyncio.run(main())  # Now visit the playground at http://localhost:8800

Need to escalate to a human? Chain logic? Prioritize some behaviors over others? All covered with shorthand guideline relationship creation:

await transfer_to_human_sales_rep.prioritize_over(offer_on_sale_car)

🧰 Tools, Utterances, and Real Conversations​

You can also easily register conditionally-evaluated tools with journeys, and add utterance templates directly:

@p.tool
async def get_on_sale_car(context: p.ToolContext) -> p.ToolResult:
    return p.ToolResult(
        data="Toyota Prius",
        utterance_fields={"car_on_sale": "Toyota Prius"},
    )

await my_journey.attach_tool(
    tool=get_on_sale_car,
    condition="the customer is interested in budget options",
)

await my_agent.create_utterance(
    template="The {{car_on_sale}} is currently on sale! Want to hear more?"
)

And because utterances are separate from generation, you get fluidity with control — so your messages stay accurate, on-brand, and hallucination-free.

🛠️ Parlant is Now a True Agent Framework​

This release makes Parlant more than a config-driven system. It’s a developer-friendly framework for building scalable, interpretable, and enforceable agent behavior — all in Python.

You get:

• A clean async API.
• Strong defaults with deep extensibility.
• Full parity with CLI configuration (and more coming).

📦 Getting Started​

To try it out, install the SDK:

pip install parlant

Then head to the Introduction Docs to get started.

🧭 What's Next​

With 2.2, Parlant is no longer just a runtime — it’s the foundation for your conversational agent's logic, end to end.

We can’t wait to see what you build.

— Team Parlant

Get in touch with us

TL;DR Conversational AI has transformed from ELIZA's simple rule-based systems in the 1960s to today's sophisticated platforms. The journey progressed through scripted bots in the 80s-90s, hybrid ML-rule frameworks like Rasa in the 2010s, and the revolutionary large language models of the 2020s that enabled natural, free-form interactions. Now, cutting-edge conversation modeling platforms like Parlant combine LLMs' generative power with structured guidelines, creating experiences that are both richly interactive and practically deployable—offering developers unprecedented control, iterative flexibility, and real-world scalability.

ELIZA: The Origin of Conversational Agents (1960s)​

The lineage of conversational AI begins with ELIZA, created by Joseph Weizenbaum at MIT in 1966.

ELIZA was a rule-based chatbot that used simple pattern matching and substitution rules to simulate conversation. Weizenbaum’s most famous script for ELIZA, called “DOCTOR,” parroted a Rogerian psychotherapist: it would reflect the user’s inputs back as questions or prompts. For example, if a user said “I feel stressed about work,” ELIZA might reply, “Why do you feel stressed about work?” This gave an illusion of understanding without any real comprehension of meaning.

ELIZA was one of the first programs to attempt the Turing Test (engaging in dialogue indistinguishable from a human). While it was a very simple system, ELIZA proved that humans could be momentarily convinced they were chatting with an understanding entity – a phenomenon later dubbed the “Eliza effect.” This early success sparked widespread interest and laid the foundation for chatbot development, even though ELIZA’s capabilities were rudimentary and entirely scripted.

Scripted Chatbots: Menu-Driven Systems and AIML (1980s–1990s)​

After ELIZA, conversational systems remained largely rule-based but grew more sophisticated.

Many early customer service bots and phone IVR systems in the 1980s and 1990s were essentially menu-driven – they guided users through predefined options (e.g. “Press 1 for account info, 2 for support”) rather than truly “understanding” free text.

Around the same time, more advanced text-based bots used bigger rule sets and pattern libraries to appear conversational. A landmark was A.L.I.C.E. (Artificial Linguistic Internet Computer Entity), introduced in 1995 by Richard Wallace. ALICE employed a specialized scripting language called AIML (Artificial Intelligence Markup Language) to manage conversation rules. Instead of hard-coding every response, AIML let developers define patterns and template replies. As a result, ALICE had an enormous base of about 41,000 predefined templates and pattern-response pairs. This allowed it to engage in more varied, natural-sounding chats than ELIZA’s simple keyword tricks. ALICE was even awarded the Loebner Prize (a conversational AI contest) multiple times in the early 2000s.

Despite these improvements, bots like ALICE and its contemporaries still relied on static scripts. They lacked true understanding and could be easily led off-track by inputs outside their scripted patterns. In practice, developers often had to anticipate countless phrasings or guide users to stay within expected inputs (hence the popularity of menu-driven designs for reliability). By the late 1990s, the paradigm in industry was that chatbots were essentially expert systems: large collections of if-then rules or decision trees. These systems worked for narrowly defined tasks (like tech support FAQs or simple dialog games) but were brittle and labor-intensive to expand. Still, this era demonstrated that with enough rules, a chatbot could handle surprisingly complex dialogues – a stepping stone toward more data-driven approaches.

The Rise of ML and Hybrid NLU Frameworks (2010s)​

The 2010s saw a shift toward machine learning (ML) in conversational AI, aiming to make chatbots less brittle and easier to build. Instead of manually writing thousands of rules, developers began using statistical Natural Language Understanding (NLU) techniques to interpret user input.

Frameworks like Google’s Dialogflow and the open-source Rasa platform (open-sourced in 2017) exemplified this hybrid approach. They let developers define intents (user’s goals) and entities (key information), and then train ML models on example phrases. The ML model generalizes from those examples, so the bot can recognize a user request even if it’s phrased in an unforeseen way. For instance, whether a user says “Book me a flight for tomorrow” or “I need to fly out tomorrow,” an intent classification model can learn to map both to the same “BookFlight” intent. This significantly reduced the need to hand-craft every possible pattern.

Over time, these NLU models incorporated Transformer-based innovations to boost accuracy. For example, Rasa introduced the DIET (Dual Intent and Entity Transformer) architecture, a lightweight transformer network for intent classification and entity extraction. Such models approach the language-understanding performance of large pre-trained transformers like BERT, but are tailored to the specific intents/entities of the chatbot. Meanwhile, the dialogue management in these frameworks was still often rule-based or followed story graphs defined by developers. In Dialogflow, one would design conversational flows with contexts and transitions. In Rasa, one could write stories or rules that specify how the bot should respond or which action to take next given the recognized intent and dialogue state.

This combination of ML + rules was a major step up. It allowed chatbots to handle more natural language variation while maintaining controlled flows for business logic. Many virtual assistants and customer support bots deployed in the late 2010s (on platforms like Facebook Messenger, Slack, or bank websites) were built this way. However, challenges remained. Designing and maintaining the conversation flows could become complex as an assistant’s scope grew. Every new feature or edge case might require adding new intents, more training data, and more dialogue branches – which risked turning into a tangle of states (a “graph-based” framework that can become overwhelmingly complex as the agent grows).

Moreover, while these systems were more flexible than pure rules, they still could fail if users went truly off-script or asked something outside the trained data.

The LLM Era: Prompt-Based Conversations and RAG (2020s)​

A watershed moment came with the advent of Large Language Models (LLMs) in the early 2020s. Models like OpenAI’s GPT-3 (2020) and later ChatGPT (2022) demonstrated that a single, massive neural network trained on internet-scale data could engage in remarkably fluent open-ended conversations.

ChatGPT, for instance, can generate responses that are often difficult to distinguish from human-written text, and it can carry on a dialogue spanning many turns without explicit rules scripted by a developer. Instead of defining intents or writing dialogue trees, developers could now provide a prompt (e.g. a starting instruction like “You are a helpful customer service agent...”) and let the LLM generate the conversation. This approach flips the old paradigm: rather than the developer explicitly mapping out the conversation, the model itself learned conversational patterns from its training data and can dynamically produce answers.

However, using LLMs for reliable conversational agents brought new challenges. Firstly, large models have a fixed knowledge cutoff (ChatGPT’s base knowledge, for example, only went up to 2021 data in its initial release). And they are prone to “hallucinations” – confidently generating incorrect or fabricated information when asked something outside their knowledge.

To tackle this, a technique called Retrieval-Augmented Generation (RAG) became popular. RAG pairs the LLM with an external knowledge source: when a user asks a question, the system first retrieves relevant documents (from a database or search index) and then feeds those into the model’s context so it can base its answer on up-to-date, factual information. This method helps address the knowledge gap and reduces hallucinations by grounding the LLM’s responses in real data. Many modern QA bots and enterprise assistants use RAG – for example, a customer support chatbot might retrieve policy documents or user account info so that the LLM’s answer is accurate and personalized.

Another tool in this era is the use of system prompts and few-shot examples to steer LLM behavior. By providing instructions like “Always respond in a formal tone,” or giving examples of desired Q&A pairs, developers attempt to guide the model’s style and compliance with rules. This is powerful but not foolproof: LLMs often ignore instructions if a conversation is long or if the prompt is complex, as parts fall out of its attention.

Essentially, pure prompting lacks guarantees – it’s still the model’s learned behavior that decides the outcome. And while RAG can inject facts, it “can’t guide behavior” or enforce complex dialogue flows. For instance, RAG will help a bot cite the correct price from a database, but it won’t ensure the bot follows a company’s escalation protocol or keeps a consistent persona beyond what the prompt suggests.

By late 2024, developers had a mix of approaches for conversational AI:

• Fine-tuning an LLM on custom data to specialize it (which can be expensive and inflexible, often requiring re-training the whole model for small changes).
• Prompt engineering and RAG to leverage pre-trained LLMs without full retraining (quick to prototype, but needing careful tweaking and still lacking strong runtime control and consistency).
• Traditional frameworks (intents/flows or graphical dialog builders) which offer deterministic behavior but at the cost of flexibility and significant manual work, especially as complexity grows.

Each approach had trade-offs. Many teams found themselves combining methods and still encountering issues with consistency and maintainability. This set the stage for a new paradigm aiming to capture the best of both worlds – the knowledge and linguistic fluency of LLMs with the control and predictability of rule-based systems. This emerging paradigm is what we refer to as Conversation Modeling.

Conversation Modeling with Parlant: A New Paradigm​

The latest development in conversational AI is the rise of Conversation Modeling platforms, with Parlant as a prime example. Parlant is an open-source Conversation Modeling Engine designed to build user-facing agents that are adaptive, yet predictable and accurate. In essence, it provides a structured way to shape an LLM-driven conversation without reverting to rigid workflows or expensive model retraining. Instead of coding up dialogue flows or endlessly tweaking prompts, a developer using Parlant focuses on writing guidelines that direct the AI’s behavior.

Get in touch with us

Guideline-Driven Conversations​

Guidelines in Parlant are like contextual rules or principles that the AI agent should follow. Each guideline has a condition (when it applies) and an action (what it should make the agent do).

For example, a guideline might be: When the user is asking to book a hotel room and they haven’t specified the number of guests, then ask for the number of guests. This “when X, then Y” format encapsulates business logic or conversation policy in a flexible, declarative way. The crucial difference from old-school rules is that guidelines don’t script out the exact wording of the bot’s response or a fixed path – they simply set expectations that the generative model must adhere to.

Parlant’s engine takes care of enforcing these guidelines during the conversation. It does so by dynamically injecting the relevant guidelines into the LLM’s context at the right time.

In our hotel booking example, if the user says, “I need a hotel in New York this weekend,” Parlant would recognize that the “ask about number of guests” guideline's condition is met. It would then load that guideline into the prompt for the LLM, so the AI’s response would be guided to, say, “Certainly! I can help with that. How many guests will be staying?” instead of the model’s default response which might have omitted the guest count question. If another guideline says the agent should always respond enthusiastically, that guideline would also be activated, ensuring the tone is upbeat. This way, multiple guidelines can shape each response.

Importantly, Parlant keeps the model’s “cognitive load” light by only including guidelines that are contextually relevant, given the current conversation state. An agent could have dozens of guidelines defined, but the user doesn’t get bombarded with irrelevant behavior – the system is smart about which rules apply when.

This dynamic approach allows richer interactions than a static flowchart: the conversation can go in many directions, but whenever a situation arises that has a guideline, the model will consistently follow that instruction. In effect, the LLM becomes more grounded and consistent in its behavior, without losing its natural language flexibility.

Reliability, Enforcement, and Explainability​

A standout feature of Parlant’s conversation modeling is how it checks and explains the agent’s decisions.

Traditional chatbots might log which intent was matched or which rule fired, but Parlant goes further. It actually supervises the AI’s output before it reaches the user to ensure guidelines were followed. One novel technique the Parlant team developed is called Attentive Reasoning Queries (ARQs).

In simplified terms, ARQs are an internal query the system poses (via the LLM’s reasoning capabilities) to double-check that the response satisfies the active guidelines. If something is off – say the model produced an answer that violates a guideline or contradicts a prior instruction – Parlant can catch that and correct course. This might involve instructing the model to try again or adjusting the context. The result is an extra layer of assurance that the agent’s answers are on-policy and safe before the user sees them.

From a developer’s perspective, this yields a high degree of predictability and makes it easier to debug conversations. Parlant provides extensive feedback on the agent’s decisions and interpretations. One can trace which guideline triggered at a given turn, what the model “thought” the user meant, and why it chose a certain reply.

This level of transparency is rarely available in pure LLM solutions (which can feel like a black box) and even in many ML-based frameworks. If a conversation went wrong, you can quickly see if a guideline was missing or mis-specified, or if the AI misunderstood because no guideline covered a scenario – and then adjust accordingly.

Get in touch with us

Faster Iteration and Scalable Testing​

Conversation modeling also dramatically improves the development lifecycle for AI agents. In older approaches, if a business stakeholder said “Our chatbot should change its behavior in X scenario,” implementing that could mean re-writing parts of a flow, collecting new training data, or even fine-tuning a model – and then testing extensively to ensure nothing else broke. With Parlant, that request usually translates to simply adding or editing a guideline.

For instance, if the sales team decides that during holidays the bot should offer a 10% discount, a developer can implement a guideline: When it is a holiday, then the agent should offer a discount. There’s no need to retrain the language model or overhaul the dialog tree; the guideline is a modular addition.

Parlant was built so that developers can iterate quickly in response to business needs, updating the conversational behavior at the pace of changing requirements. This agility is akin to how a human manager might update a customer service script or policies and immediately all agents follow the new policy – here the “policies” are guidelines and the AI agent follows them immediately once updated.

Because guidelines are discrete and declarative, it’s also easier to test and scale conversational agents built this way. Each guideline can be seen as a testable unit: one can devise example dialogues to verify the guideline triggers properly and the agent’s response meets expectations. Parlant’s deterministic injection of guidelines means the agent will behave consistently for a given scenario, which makes automated testing feasible (you won’t get a completely random response every time, as raw LLMs might give).

The platform’s emphasis on explainability also means you can catch regressions or unintended effects early – you’ll see if a new guideline conflicts with an existing one, for example. This approach lends itself to more robust, enterprise-grade deployments where reliability and compliance are crucial.

Integration with Business Logic and Tools​

Another way Parlant stands apart is in how it separates conversational behavior from back-end logic.

Earlier chatbot frameworks sometimes entangled the two – for example, a dialog flow node might both decide what to say and invoke an API call. Parlant encourages a clean separation: use guidelines for conversation design, and use tool functions (external APIs or code) for any business logic or data retrieval.

Guidelines can trigger those tools, but they don’t contain the logic themselves. This means you can have a guideline like “When the customer asks to track an order, then retrieve the order status and communicate it.”

The actual act of looking up the order status is done by a deterministic function (so no uncertainty there), and the guideline ensures the AI knows when to call it and how to incorporate the result into the conversation. By not embedding complex computations or database queries into the AI’s prompt, Parlant avoids the pitfalls of LLMs struggling with multi-step reasoning or math.

The division of labor leads to more maintainable and reliable systems: developers can update business logic in code without touching the conversation scripts, and vice versa. It’s a design paradigm that scales well as projects grow.

Real-World Impact and Use Cases​

All these capabilities make conversation modeling suitable for applications that were previously very challenging for conversational AI.

Parlant emphasizes use cases like regulated industries and high-stakes customer interactions. For example, in financial services or legal assistance, an AI agent must strictly follow compliance guidelines and wording protocols – a single off-script response can have serious consequences. Parlant’s approach ensures the agent reliably follows prescribed protocols in such domains.

In healthcare communications, accuracy and consistency are paramount; an agent should stick to approved responses and escalate when unsure. Guidelines can encode those requirements (e.g. “if user mentions a medical symptom, always provide the disclaimer and suggest scheduling an appointment”).

Brand-sensitive customer service is another area: companies want AI that reflects their brand voice and policies exactly. With conversation modeling, the brand team can literally read the guidelines as if they are a policy document for the AI. This is a big improvement over hoping an ML model “learned” the desired style from training examples.

Teams using Parlant have noted that it enables richer interactions without sacrificing control. Users aren’t forced down rigid conversational menus; instead, they can ask things naturally and the AI can handle it, because the generative model is free to respond creatively as long as it follows the playbook defined by guidelines.

At the same time, the development overhead is lower – you manage a library of guidelines (which are human-readable and modular) instead of a spaghetti of code. And when the AI does something unexpected, you have the tools to diagnose why and fix it systematically.

In short, Parlant’s conversation modeling represents a convergence of the two historical threads in chatbot evolution: the free-form flexibility of advanced AI language models with the governed reliability of rule-based systems. This paradigm is poised to define the next generation of conversational agents that are both intelligent and trustworthy, from virtual customer assistants to automated advisors across industries.

Get in touch with us

TL;DR Eliminate LLM hallucinations and gain full control over chat style and tone of voice with our new Utterance Templates.

At Parlant, we're committed to pushing the boundaries of conversational AI, ensuring that interactions are not only intelligent but also consistent, compliant, and aligned with your brand's voice.

Today, we're thrilled to unveil a game-changing feature: Utterance Templates.

🧠 What Are Utterance Templates?​

Utterance Templates empower you with precise control over your Parlant agent's responses. By restricting outputs to a predefined set of responses, you ensure your agent communicates with a consistent tone, style, and accuracy, aligning perfectly with your brand voice and service protocols while completely eliminating the risk of even subtle unwanted or hallucinated outputs.

Think of it as handing your agent a curated set of response cards. Based on the conversation context, the agent selects the most appropriate "card" to play, ensuring every interaction is both relevant and on-brand.

🛠️ How It Works​

Under the hood, Utterance Templates operate through a three-stage process:

1. Draft Message Generation: The agent drafts a fluid message based on the current situational awareness, including interaction history, guidelines, and tool results.

2. Template Matching: It then matches the closest utterance template from your predefined set.

3. Rendering: The engine renders the matched template using the Jinja2 templating engine, supporting variable substitutions using tool results, and generative substitutions when explicitly allowed.

This process ensures that responses are not only contextually appropriate but also adhere strictly to your specified guidelines.

🧪 Real-World Use Case: Streamlining Customer Support​

Scenario: A customer inquires about product availability.

Without Utterance Templates:

Customer: Do you have it in stock?

Agent: Yes, we've got this item in stock! Let me know if you need any help finding it.

With Utterance Templates:

Customer: Do you have it in stock?

Agent: Yeah bro. Should I add it to your cart?

By utilizing Utterance Templates, the agent's response is not only concise but also aligns with the brand's conversational style, ensuring consistency across interactions.

Get in touch with us

🔒 Why This Matters: Enhancing Compliance and Brand Integrity​

In industries where compliance and brand consistency are paramount—such as finance, healthcare, and legal services—Utterance Templates offer unparalleled advantages:

• Elimination of Hallucinations: By restricting responses to predefined templates, you eliminate the risk of AI-generated inaccuracies.

• Regulatory Compliance: Ensure that all communications adhere to industry regulations and standards.

• Brand Consistency: Maintain a uniform tone and style across all customer interactions, reinforcing brand identity.

🚀 Getting Started​

Ready to take control of your conversational AI? Here's how to enable Utterance Templates:

1. Define Your Templates: Create a set of response templates using the Jinja2 engine, incorporating variable substitutions as needed.

2. Set Composition Mode: Choose the appropriate composition mode for your use case:

parlant agent update \
  --id $AGENT_ID \
  --composition-mode $COMPOSITION_MODE  # [strict-utterance|fluid-utterance]

• Strict Utterance: Agent can only select responses from defined templates.
• Fluid Utterance: Agent prioritizes templates but may fall back to default message generation if no good match is found.

3. Monitor and Refine: Use Parlant's integrated UI to inspect draft messages and refine templates for optimal performance.

For a detailed guide, visit our Utterance Templates Documentation.

💬 Join the Conversation​

We believe Utterance Templates will revolutionize the way businesses approach conversational AI, offering a blend of control, compliance, and customer satisfaction. We'd love to hear how you're planning to implement this feature in your workflows.

Reach out to our team for personalized assistance.

Here's to more meaningful and consistent conversations!

Get in touch with us