Agentic Backends: What I Wish Someone Had Told Me About API Design for LLMs
Modern LLMs APIs have this powerful feature called "tool calling" or "function calling". Essentially, they can call API endpoints or functions to perform actions in the real world. Send an email, update a database, fetch weather data, etc.
Naturally, when developers discover this, their first reaction is: "Perfect! We already have an API. We'll just expose our existing endpoints as tools and let the LLM use them." This is seen as a quick and elegant way toward a conversational UI for your APIs.
Except it doesn't really work.
The Core Issue
While LLMs are essentially API consumers, they're fundamentally different from normal ones, which are... human developers. When an informed developer uses an API, they bring a lot of implicit knowledge to the table:
- Familiarity with your system's concepts and entities
- Understanding of the business logic and purpose
- Knowledge of the API's modularity and flow
But with LLMs, even if you feed them with some context, they're (comparatively) flying blind.
The problem is further amplified by the fact that LLMs are trained on billions of pages of human text like stories, conversations, and articles. So they're incredibly good at understanding natural language patterns. But APIs live in a different semantic universe. They contain numerous idiosyncracies that will trip up even the most sophisticated models.
Even more importantly, most APIs have zero tolerance for mistakes of even the smallest kind. It's not enough to be "semantically similar" when filling out a parameter. You have to follow a strict contract with absolute consistency.
Fred Brooks put it best in his classic book, The Mythical Man-Month: “If one character, one pause, of the incantation is not strictly in proper form, the magic doesn’t work [...] Adjusting to the requirement for perfection is, I think, the most difficult part of learning to program.”
Modularity is a Double-Edged Sword
Perhaps most critically, traditional APIs are modular by design. Want to send an email to a user? First, fetch the user's details. Then validate their email preferences. Finally, call the send endpoint. Each step has its own error states and success conditions.
For a human developer, this modularity is a crucial feature. For an LLM, it's literally begging for trouble.
We've consistently watched LLMs:
- Call step 3 without doing steps 1 and 2
- Repeat step 1 endlessly because they forgot they already did it
- Invent parameters for step 2 based on a creative interpretation of the function name
- Pass arguments that break the type system or violate the API contract
- Give up entirely and hallucinate a success response
Time for a Different Approach
Many are working on making LLMs smarter about API interpretation and coding. This is a good step, but we need to do more. It's not enough to improve function-calling quality by another few percentage points.
We need to rethink how we design for LLMs. We need to understand that LLMs are a different type of API consumer with different needs, and design our APIs accordingly.
Essentially, what I'm suggesting is applying the pattern of BFF or Backend for Frontend, but to LLM (or agentic) frontends.
In other words, instead of creating a single API that categorically serves all consumers, we create a specialized API that is tailored to the unique needs of agentic frontends.
With that in mind, let's look at some of the most common failure modes and how to avoid them when designing APIs for LLMs.
Get in touch with us
1. Tools Need to Be Standalone
Traditional API design embraces the principle of composability: small, reusable functions that work together. Need to update a user? First fetch their ID, then call the update endpoint.
But LLMs work best when they can match a clear user intent to a single action.
Consider these two API designs:
Traditional (Bad for Agents):
def get_user_id(email: str) -> int: ...
def update_user_preferences(user_id: int, preferences: dict) -> bool: ...
Agent-Friendly (Good):
def update_user_preferences_by_email(email: str, preferences: dict) -> bool: ...
Agents struggle with multi-step workflows. Each tool call is a decision point where things can go wrong. The agent might:
- Forget to call
get_user_id() - Forget to store the intermediate result
- Misunderstand which ID to use if multiple are in context
- Hallucinate a user ID based on partial information
- Lose track of the overall goal between calls
Key Principle: Between "call the same function but parameterize it differently" and "create a standalone function for this particular context", try to heavily lean toward the latter. Think of a tool as a unique UI button instance.
Avoiding Ambiguity
A key observation here is that LLM APIs will naturally contain a lot of redundancy. This is not a bug, it's a feature. The more you can make each tool call self-contained, the less cognitive load you place on the agent, and the more robust your agentic API will be. See what I mean when I say we need to think differently about agentic APIs?
Additionally, tool functions will sometimes differ from each other in only minor ways. For example, schedule_appointment() and reschedule_appointment(). In a human context, these are clear. But for an agent, they can be confusing. This means that you should try to control, to the best of your ability, what tools you even allow your LLM agent to evaluate at any given time.
Incidentally, this is exactly why, in Parlant, tools must be associated with guidelines and can only run when the conditions for their associated guidelines are active.
2. Tools Should Be Idempotent
Like I said before, LLMs will often call the same tool multiple times during a single interaction, either because they forgot they already did it or because they mistakenly think they need to do it again.
For example, they'll mistakenly call make_transaction() or send_email() multiple times, causing a mess. So if a customer comes in and says, "I want to send $50 to Bill", the agent then calls make_transaction(amount=50, recipient="Bill"). Then the customer says, "Thanks, so it's done, right?" and the agent calls it again, thinking it needs to confirm the transaction.
And there are even more trivial examples like just calling a tool again because the context still kind of "makes sense."
This is why you should design your tools to be idempotent. In other words, calling them multiple times with equivalent parameters (equivalent being an important word we'll circle back to) should yield the same result without side effects.
In Parlant, one of the tricks you can use to achieve this is to use the session ID as an idempotence key.
import parlant.sdk as p
@p.tool
def make_transaction(
context: p.ToolContext,
amount: float,
recipient: str,
) -> p.ToolResult:
global TRANSACTION_IDEMPOTENCE_TABLE
if context.session_id in TRANSACTION_IDEMPOTENCE_TABLE:
return p.ToolResult("Transaction already made in this session")
# Proceed with the transaction logic
The approach above is more simplistic and aggressive, essentially allowing only a single transaction per session. Of course, there are more nuanced ways to approach it, but we'll leave it as an exercise for the reader.
Get in touch with us
3. Names Matter More Than You Think
Quick: what's the difference between these functions?
send_message()send_notification()send_email()send_alert()
To human developers reading docs it may be crystal clear, because they understand the API's terminology, and that each entity is a fundamentally different creature.
To an LLM it's a minefield. You won't believe how many false positives it'll have here.
Bad Design:
def send_notification(user_id: str, title: str, message: str): ...
def send_message(recipient: str, content: str): ...
Good Design:
def send_email_to_user(email_address: str, subject: str, body: str):...
def send_phone_sms_to_user(phone_number: str, text_content: str): ...
Be tediously explicit. Use names that:
- Include the key differentiator in the function name itself
- Clearly indicate what type of data is expected
- Avoid synonyms that might confuse
- Attract contextual attention with literal clues
4. Parameters Are Queries in Disguise
With regular APIs, you can often assume the developer abided by the contract and passed the right data. For example, they won't pass an email for user_id if they can see that the API has a UserID entity that's returned by other endpoints. If they do, you just return an error. And once they see that their code is broken, they fix it. Then they ship it to production, and everything works.
But LLMs don't have that luxury, and they keep trying to figure out your API, from scratch, every single time. They won't code and debug it until it works, and only then ship it.
So if you don't want your agent to consistently fail at tool calls, you need to provide tool functions that are as tolerant as possible toward their arguments.
An effective design pattern here is to treat every parameter in your tool as essentially a query for that parameter's value. Sometimes the translation is direct, and sometimes it requires some inference.
Consider this function:
def schedule_meeting(participant_1, participant_2):
If each participant must follow a strict schema, such as email or valid user ID, then, for each parameter, the agent must:
- Determine if it has this information
- Extract it from context if available
- Decide on a reasonable default if not
- Format it correctly
That's 8 potential failure modes right there.
Instead, you can think of each parameter as a query. Something like this:
import parlant.sdk as p
@p.tool
def schedule_meeting(
context: p.ToolContext,
participant_1: str | None = None,
participant_2: str | None = None,
) -> p.ToolResult:
participants = [participant_1, participant_2]
if None in participants:
return p.ToolResult("Both participants must be provided")
actual_users = []
for p in participants:
user = None
# infer_type classifies with regex or perhaps even with NLP models
match infer_type(participant_1):
case "email":
user = find_user_by_email(p):
case "user_id":
user = find_user_by_user_id(p)
case "full_name":
user = find_user_by_full_name(p)
case _: pass
if user is None:
return p.ToolResult(f"Could not find user for {p}")
# Proceed with scheduling the meeting using actual_users
This will produce much more consistency in production.
Agent-Friendly APIs
These agent-friendly APIs are more verbose. They have more endpoints. They're more opinionated.
But we've learned that the cost of an agent misusing an API is far higher than the cost of a slightly larger API surface. One bad function call can derail an entire agent interaction, frustrating users and destroying trust.
Practical Checklist for Agentic API Design
Before releasing an API for agent consumption, ask yourself:
- Can each tool facilitate a user's intent in a single, standalone call?
- Would a non-technical person understand non-ambiguously what each tool does from its name alone?
- Is each parameter name specific enough that misinterpretation is highly improbable?
- Are your tools built to be idempotent, so they can be called multiple times without risky side effects?
- Are you treating parameters as queries, allowing for flexible input without strict adherence to narrow contracts?
Conclusion
As AI agents become more prevalent, we need to evolve our API design philosophy. The principles that served us well for human-consumed APIs—flexibility, composability, abstraction—can actually become liabilities in the agentic world.
The good news is that LLMs are incredibly powerful and actually quite consistent when we design for their characteristics.
Have you encountered other surprising challenges in agentic API design? I'd love to hear about them.
Get in touch with us