Let me save you a LinkedIn scroll: most things people call "AI agents" are chatbots wearing a trench coat. They take your input, generate some text, and hand it back with the confidence of someone who has never once checked a database.
I've been running actual production AI agents - the kind that call APIs, query databases, and take real actions - for over three years. Here's what that distinction costs you when you ignore it.
The Text-In-Text-Out Trap
The majority of LLM applications today work like this: you send text, you receive text. Impressive text, sure. Eloquent, even. But fundamentally, it's a closed loop. The model has no hands. It can describe how to check your inventory with remarkable clarity, but it cannot actually check your inventory.
This is the difference between an agent and a chatbot. A chatbot tells you what the weather might be. An agent checks the weather API and tells you it's 14°C and you should bring a jacket. One is helpful. The other is literature.
"Just Add Tool Use" - Famous Last Words
When we built our first agents using LangChain, MCP didn't exist. There was no standard protocol for tool access. There was us, some JSON schemas, and a disturbing amount of YAML.
We had to design everything from scratch: how tools describe themselves to the model, how the model requests a tool call, how results get serialized back, what happens when a tool times out, what happens when the model hallucinates a tool that doesn't exist (spoiler: this happens more than you'd like).
It was, in the most generous interpretation, "character building."
What Actually Makes Tool Use Hard
Calling a function isn't hard. A junior developer can wire up an API call in an afternoon. The hard part is everything else:
Tool selection at scale. Give a model 3 tools and it picks the right one most of the time. Give it 15 tools and suddenly it's calling the weather API when you asked about inventory. The model doesn't just need tools - it needs judgment about which tool to use, and that judgment degrades as the toolbox grows.
Parameter extraction from vibes. A user says "show me something similar but in blue." The model needs to turn that into a structured API call with color=blue and similar_to=<previous_product_id>. This is not a parsing problem. This is a mind-reading problem, and the model is occasionally wrong in creative ways.
Graceful failure. Tools fail. APIs time out. Databases return empty results. Auth tokens expire mid-conversation. The model needs to handle all of this without telling the user "an unexpected error occurred" - which, by the way, is the model equivalent of shrugging.
Multi-step chains where step 3 depends on step 1. Real tasks rarely need one tool call. They need a sequence: look up the customer, check their order history, find the relevant product, check stock levels, generate a recommendation. If step 2 fails, the model needs to figure out what to do - not just crash with a confused emoji.
The "Agent" Litmus Test
Here's a simple test: if you remove the LLM and replace it with a hardcoded script, does the system still work the same way? If yes, it was never an agent. It was an automation workflow with a chatbot UI.
A real agent makes decisions the script couldn't predict. It reasons about which tools to call, adapts when things don't go as planned, and produces outputs that depend on the specific context of the conversation. The LLM isn't decoration - it's the decision engine.
What I'd Do Differently Today
If I were starting fresh, I'd adopt MCP from day one. Not because our custom tool interfaces didn't work - they've been running in production for three years and they're fine - but because I've spent enough time debugging custom serialization to last a lifetime.
MCP standardizes the boring parts (tool schemas, transport, error handling) so you can focus on the interesting parts (what tools to build, how to orchestrate them, when to let the agent act autonomously vs. when to check with a human).
That said, the principles we learned the hard way still apply no matter what protocol you use: clear tool schemas, explicit error contracts, always have a fallback, and never trust the model to pick the right tool on the first try without testing it extensively.
The Uncomfortable Truth
Most companies don't need agents. They need better automations with a conversational UI. And that's fine. Not everything needs to be autonomous.
But if you do need an agent - if your use case genuinely requires reasoning, tool selection, and multi-step execution - then tool use isn't a feature. It's the whole point. Without it, you're just paying for a very expensive text generator that sounds confident about things it has never verified.
And confidence without verification? We already have enough of that in most organizations.