The question I’m wrestling with is will anybody care about MCP? I’m working on my own MCP proxy to manage security, auditing, and server management and the more I think deeply about the actual use cases the more I wonder if I’m wasting my time.
Can anyone think of a world where MCP is relevant if generic chatbots (ChatGPT, Claude Desktop) don’t become the primary human-AI interface? If LLMs are still wrapped in application wrappers, isn’t ̶a̶n̶ ̶a̶p̶p̶r̶o̶a̶c̶h̶ ̶l̶i̶k̶e̶ ̶L̶a̶n̶g̶C̶h̶a̶i̶n̶ a more traditional agentic approach going to make more sense?
I think MCP has legs well beyond just the LLM / agent world. Just like USB went from "how I connect my mouse" to "how I charge my beard trimmer."
In fact, I imagine it's going to go full-duplex with all our systems, becoming a more standard way for systems to communicate with each other.
Under the hood, MCP is just JSON RPC, which is a fine format for communicating between systems.
MCP layers on some useful things like authentication and discovery. Both are critical to any kind of communication between systems built by different authors (e.g. various apps and services). Discovery, especially, is the fascinating part. Rather than hoping an OpenAPI spec exists and hoping it's right, MCP has this exchange of capabilities baked in.
I spent the last 9 years building integration technology, and from that perspective, the discovery-documentation-implementation problem is the core issue.
Right now, LLMs basically "solve" the integration problem because they can do the mapping between external tools/resources/formats and internal ones.
But there's nothing that strictly "requires" an LLM to be involved at all. That's just the primary reason to develop MCP. But you could just as well use this as a way for integrating systems, making some bets on interface stability (and using LLMs for cases only when your prior expectations no longer hold and you need a new mapping).
The comparison is perhaps imperfect and overused, but I feel like we're witnessing the birth of a new USB-like standard. There's something right now that it was designed to do, but it's a decent enough standard that can actually handle many things.
I wouldn't be surprised if in some period of time we see enterprise apps shift from REST to MCP for bi-directional integrations.
For the OP, I'm not sure if you're working on an MCP proxy (A) as a commercial offering, (B) as something for your team to use, closed source, or (C) as something open source for fun. But we just built and started selling an MCP proxy/gateway. It handles identities for humans & bots, tool allowlists, and policy setting for an org.
If you don't want to build something on your own because of option B above, get in touch.
It could, except LLMs are non-deterministic, and the rest of the tech world largely isn't, and aligning those two and keeping them aligned with every tweak to a model and model change, and prompt change can be a lot of babysitting.
No doubt there's lots of smart people working on it, I just have been around application of tech in B2B for reliability and that's usually where the conversation usually starts.
A concise, no-nonsense list of every endpoint your service offers, with a simple text description and a JSON schema, is all that many developers ever wanted and needed in a lot of cases, but no one bothered producing them until we invented a machine that could automatically make use of them.
In my opinion, I get the desire to create some sort of specification for an LLM to interface with [everything else], but I don't really see the point at doing it on an inference level by smashing JSON into the context.
These models are usually very decent at parsing out stuff like that anyway; we don't need the MCP spec, everyone can just specify the available tools in natural language and then we can expect large param models to just "figure it out".
If MCP had been a specification for _training_ models to support tool use on an architectural level, not just training it to ask to use a tool with a special token as they do now.
It's an interesting topic because it's the exact same as the boundary between humans (sloppy, organic, analog messes) and traditional programs (rigid types, structures, formats).
To be fair if we can build tool use in architecturally and solve the boundary between these two areas then it also works for things like objective facts. LLMs are just statistical machines and data in the context doesn't really mean all that much, we just hope it is statistically relevant given some input and it is often enough that it works, but not guaranteed.
> These models are usually very decent at parsing out stuff like that anyway; we don't need the MCP spec, everyone can just specify the available tools in natural language and then we can expect large param models to just "figure it out".
This is mostly the kind of misunderstanding of MCP that the article seems directed at, and much of this response is focussed on things that are key points in the article, but:
MCP isn't for the models, it is for the toolchains supporting them. The information models actually need about tools and resources is accessed from the server by the toolchain using the information that is in the MCP, and the structure that models use varies by the model, but it is consistently completely different information than what is in the MCP—the tool and resource (but probably not prompt) names from the MCP will probably also be given to the model, but that's pretty much the only direct overlap. MCP can also define prompts for the toolchain, but information about those are more likely presented directly to the user than the model itself.
The toolchain also needs to know how the model is trained to get tool information in its prompt, just like it needs to know other aspects of the models preeferred prompt template, but that is a separate concern from MCP.
> If MCP had been a specification for _training_ models to support tool use on an architectural level, not just training it to ask to use a tool with a special token as they do now.
MCP isn't a specification for training anything. MCP is a specification for providing information about tools external to the toolchain running the LLM to the toolchain. Tools internal to the toolchain don't ever use MCP because, again, MCP isn't for the model, it's for the toolchain.
You've replied multiple times specifying toolchains without explaining what they are.
I've seen for models that don't support tool defs via API that those tool defs are provided in the context (though the model is still trained for tool use, outputting the special python_call/x tokens to indicate a tool call in output).
I can see for example that MCP's own example using Anthropic uses their API/SDKs tools section as outlined here https://docs.anthropic.com/en/api/messages#body-tools. What the example does is shove the tool definition into here - this includes the full name description etc of the tool.
Quoting them "And then asked the model "What's the S&P 500 at today?", the model might produce tool_use content blocks in the response" so I imagine that behind the scenes they're _smashing it into the context_ as I already suggested; the only reason it's separate in the API is so they can type/validate it.
I don't know what this magical tool chain is but the LLM is the thing providing output based on the not so new magical concept of attention and statistics; I don't see how some separate "toolchain" piece takes the input string and somehow does a better job at selecting a tool than the model itself; unless the toolchain is itself a smaller LLM trained specifically for tool use outside of your larger multi-purpose/"knowledgable" LLM.
As I mentioned in a sibling thread, you can use that JSON structured input to constrain the LLM's output during inference so that it will only contain valid tool calls, in addition to smashing it into the context. This is valuable since it's going to be far more robust than assuming that the LLM can "figure everything out" from a natural language description.
MCP is a means of communicating information about externally-defined tools to the “application wrapper” (and your examples of “generic chatbots” are also application wrappers). Well, between the application wrapper and servers; “application wrappers” for LLMs are pretty much the motivating (but not sole) case of MCP Clients.
Without something like MCP, each application wrapper is left do do its own ad hoc wrappers for external tools (tools internal to the wrapper don’t use MCP.) With MCP, it just integrates an MCP client library, and then it can use any tool, resource, or prompt provided by any MCP server available to it.
MCP is not concatenating JSON to the context. MCP is providing JSON to the toolchain; except for the names of tools and resources, most of the information in MCP doesn't go to the model at all, the toolchain uses it to connect to tool (resource, etc.) providers and from there it gets information that it can use either in the context for the LLM or in the UI for the user, and the shape that information goes into the context for the model depends on the model and has nothing to do with MCP.
MCP, is just a way for the toolchain to get information about and communicate with external services, the model doesn't (and if this sounds like the title of the article, there is a reason) need to know about it.
Yeah but I dont have to type all that context in - not to mention if I had all that context in my hand I wouldnt need to enter it into a LLM to find out what it says.
MCP sucks because it has to be connected to a desktop client. I'd love to build some MCP-like integrations but no one on my team can use them. We use LLM's via - as you noted - other means like via Notion, via web UI, via our own API integrations. Until there is a more central way to connect these things - yeah they won't reach mass adoption.
> MCP sucks because it has to be connected to a desktop client.
No, it doesn't need to be connected to a desktop client. It is true that the original use was for connecting local tools over stdio to a desktop client, and it is currently more supported in desktop clients than others, but it now includes remote support and, e.g., ChatGPT Deep Research has support for remote MCP, but only for servers with a very specific shape.
1. A user interacting with multiple MCP servers, behind a gateway (with MCP client support) to get authentication from the user to those servers in some way (OAuth/OIDC, with PKCE, usually, sometimes token exchange), allowing out-of-band auth
2. The same, but built on identity for service accounts/native identity or something, for automation
would enable this. There’s a few SEPs open now around this.
Something like containerized apps are going to be important for security with MCPs or whatever it becomes, comes from it, or comes afterwards.
Getting in reps on thinking through these kinds of problems are valuable since LLMs are a new type of software and existing software axioms don't always fit.
What about LangChain makes more sense? It’s one of the most prematurely complex libs I’ve seen. I’m calling it right now, LangChain is going to run a mind fuck on everyone and convince people that’s actually how complicated orchestrating LLM control flow should be. The community needs to fight this framework off.
That’s besides the point. MCP servers let you discover function interfaces that you’ll have to implement yourself (in which case, yeah, what’s the point of this? I want the whole function body).
Yup exactly. It's all just state machines. Really nothing more than that.
It's like all these lang* frameworks are pretending that they can solve core deficiencies in the model, whereas most stuff is just workarounds.
We do have to glue model stuff together _somehow_ but there's no reason that it needs to be as complex as most of these frameworks are setting out to be.
> The community needs to fight this framework off.
Why? The people who been around for a while, already avoid it because they've either tried it before, or poked around in the source and then we ran away quickly. If people start using stuff without even the slightest amount of thinking beforehand, then that's their prerogative, why would it be up to the community hive-mind to "chose" what tools others should use?
Agreed except we end up with a lot of junior people in the space who learned and used only langchain, who we then have to unlearn all the langchain nonsense when we hire them. Or we grep -v langchain cvs/
My bad. I shouldn’t have mentioned LangChain here because it’s a little besides my point. What I mean is, MCP seems designed for a world where users talk to an LLM, and the LLM calls software tools.
For the foreseeable future, especially in a business context, isn’t it more likely that users will still interact with structured software applications, and the applications will call the LLM? In that case, where does MCP fit into that flow?
it separates FE and BE for agent teams just like we did with web apps. the team building your agent framework might not know the business domain of every piece of your data/api space that your agent will need to interact with. in that case, it makes sense for your differnet backend teams to also own the mcp server that your companies agent team will utilize.
Yeah I don’t know. Let’s a say a org wants to do discovery of what functions are available for an app across the org. Okay, that’s interesting. But, each team can just also import a big file called all_functions.txt.
A swagger api is already kind of like an MCP, or really any existing REST api (even better because you don’t have to implement the interface). If I wanted to give my LLM brand new functionality, all I’d have to do is define out tool use for <random_api>, with zero implementation. I could also just point it to a local file and say here are the functions locally available.
Remember, the big hairy secret is that all of these things just plop out a blob of text that you paste back into the LLM prompt (populating context history). That’s all these things do.
it separates FE and BE for agent teams just like we did with web apps. the team building your agent framework might not know the business domain of every piece of your data/api space that your agent will need to interact with. in that case, it makes sense for your differnet backend teams to also own the mcp server that your companies agent team will utilize.
Why don’t they just own a REST or RPC server? This is the part of the MCP motivation I’m not totally getting. In fact, you can prove to yourself that your LLM can hook into almost any existing REST api in a few minutes, which gives it more existing options and functionality than just about anything else as it stands now.
Things like swagger or graphql already provide you discovery.
> This is the part of the MCP motivation I’m not totally getting
Would it help you to know that the original use case of MCP was communicating information about and facilitating communication with servers that the LLM frontend would run locally and communicate with over stdio, and that remains an important use case?
Total beginner question: if the “structured software application” gives llm prompt “plan out what I need todo for my upcoming vacation to nyc”, will an llm with a weather tool know “I need to ask for weather so I can make a better packing list”, while an llm without weather tool would either make list without actual weather info OR your application would need to support the LLM asking “tell me what the weather is” and your application would need to parse that and then spit back in the answer in a chained response? If so, seems like tools are helpful in letting LLM drive a bit more, right?
If you have a weather tool available it will be in a list of available tools, and the LLM may or may not ask to use it; it is not certain that it will, but if it is a 'reasoning' model it probably will.
You need to be careful creating a ton of tools and displaying a list of all of them to the model since it can overwhelm them and they can go down rabbit holes of using a bunch of tools to do things that aren't particularly helpful.
Hopefully you would have specific prompts and tools that handle certain types of tasks instead of winging it and hoping for the best.
If you're a large org with an API that an ecosystem of other partners use then you should host a remote MCP and then people should connect LLMs to it.
The current model of someone bundling tools into an MCP and then you download and run that MCP locally feels a bit like the wrong path. Tool definitions for LLMs are already pretty standardized if things are just running locally why am I not just importing a package of tools, I'm not sure what the MCP server is adding.
That's ironic. I think local MCPs are an auth nightmare.
Just think of all those plaintext auth tokens sitting in well-known locations on your machine.
It's a black hat dream.
We'll see, but I think commercial use of local MCPs is going to be constrained to use cases that only make sense if the MCP is local (e.g. it requires local file access).
For everything else, the only commercially reasonable way to use them is going to be remote streamable HTTP MCPs running in isolated containers
And even then, you need some management and identity plane. So they're going to likely be accessed via an enterprise gateway/proxy to handle things like:
- composition -- bundling multiple MCPs into one for easier connection
- identities per-user / per-agent
- generation of rotatable tokens for headless agents
- filtering what features (tools, prompts, resources) flow through into LLM context
- basic security features, like tool description whitelisting to prevent rug pulls
MCP is only a protocol, after all. It's not meant to be a batteries-included product.
...no, MCP was always designed to be run locally, the auth mess was the result of people trying to sidestep that design intent and getting grumpy that it didn't work well (surprise, of course not)
MCP is just packaging. It's the ideal abstraction for building AI applications.
I think it provides the similar benefits of decoupling the front and back end of a standard app.
I can pick my favorite AI "front end"- whether that's in my IDE as a dev, a desktop app as a business user, or on a server if I'm running an agentic workflow.
MCP allows you to package tools, prompts, etc. in a way that works across any of those front ends.
Even if you don't plan on leveraging the MCP across multiple tools in that way- I do think it has some benefits in de-coupling the lifecycle of the tool development from the model/ UI.
The biggest challenge I have is that setting up and configuring them is a mess. I'm pretty technical and I still find configuration confusing and brittle. Especially if auth is involved.
I work in a marketing team, I would love folks to be able to use Google's Analytics MCP [1]. The idea of getting people into Google Cloud, or setting up and sharing a file with service account credentials is an absolute nightmare.
I don't think these problems can't be solved, and if remote MCPs gain adoption that alone solves a lot of the issues, but the way most MCPs are packaged and shared currently leaves A LOT to be desired.
MCP seems just like a rushed concept that Anthropic shoved out there just so they could own the standard. I've been working with it a lot lately, in Go with mcp-go[0]. Very un-intuitive at first, and I constantly ask myself why I wouldn't just write this in my own way, but admittedly it can be fun.
I've been building agents for a bit (RA.Aid OSS coding agent, now Gobii web browsing agents).
The main problem with MCP is that it just makes tools available for the agent to use. We get the best performance when there's a small set of tools and we actively prompt the agent on the best way to use the tools.
Simply making more tools available can give the agent more capabilities, but it can easily trash performance.
This is 100% a problem with the MCP spec: it does not currently provide a way to narrow what tools, and therefore context, flow into the LLM.
I don't really think there's an easy solution at the protocol level, since you can't just make the LLM say what tools it wants upfront. There's a whole discovery process during the handshake:
LLM(Host): Hi, I'm Claude Desktop, what do you offer?
MCP Server: Hi, I'm Salesforce MCP, I offer all these things: {...tools, prompts, resources, etc.}
Discoverability is one of the reasons MCP has a leg up on traditional APIs. (Sure, OpenAPI helps, but it's not quite the same thing.)
I'd be interested in hearing other recommendations or ideas, but when I saw this, I realized that the spec effectively necessitates a whole new layer exist: the gateway plane.
Basically, you need a place where the MCPs can connect & expose everything they offer. Then, via composability and settings, you can select what you want to pass through to the LLM (host), given the specific job it has.
I basically pivoted my company to start building one of these, and we're getting inundated right now.
This whole thing reminds me of the early web days, where the protocols and standards were super basic and loose, and we all just built systems and tools to fill those gaps. Just because MCP isn't "complete" doesn't mean it's not valuable. In fact, I think leaving some things to the community & commercial offerings is a great way for this tech to keep winning.
> This is 100% a problem with the MCP spec: it does not currently provide a way to narrow what tools, and therefore context, flow into the LLM.
it's not the business of MCP spec, it should be task/job level, different task may need different tools and MCP just supply entire tools it has.
The tool pickup should be the take charge by llm. moreover maybe some has `while list` to include or exclude some tools, but it should not from MCP spec.
It can, but I remain deeply unconvinced that the sub-agent architecture works as well as advertised.
The trick with any layering like this is that your end-to-end reliability is subagent_reliability * routing_agent_reliabilitty. Neither are 100% (or anywhere close to it, let's be honest), so the multiplying probabilities are still going to trash your performance.
If you get routed to the correct subagent, then subsequent performance is likely to be solid - but that's because you've taken the `routing_agent_reliability` term out of the equation.
Routing agent reliability hinges pretty heavily on the subagents themselves and how semantically or linguistically similar they are. If you have subagents that are in wildly disparate domains it may work well, but if your subagents start overlapping (or just look like they overlap) then routing accuracy is likely going straight into the dumpster. And a mis-route is catastrophic in that setup.
For very specific agents (well-established workflows that cross multiple, well-defined, non-overlapping domains) the architecture may be suitable, but in terms of the holy grail of the omni-agent (i.e., a desktop app agent suitable for general use) I suspect we'll continue running into a brick wall.
Can you elaborate on how the agents degrades from more tools? By paralysis or overuse? Isn’t this both ways a function of direction on correctly instructing which to use when? Tnx
The context window is limited. Using half your context window for tools means you have a 50% smaller context window.
On a large and complex system (not even a mini ERP system or even a basic bookkeeping system, but a small inventory mgmt system) you are going to have a few dozen tools, each with a description of parameters and return values.
For anything like an ERP system you are going to have a few thousands of tools, which probably wouldn't even fit in the context before the user supplied prompt.
This is why the only use case this far for genAI is coding: with a mere 7 tools you can do everything.
> The problem of overflowing context is solved by RAGs, though.
No, it isn't.
It's mitigated with RAGs, but RAGs add to the context, and what they add might be irrelevant is all the retriever module is doing is plain text search.
If the retriever module is performing an embeddings/vector search on a properly prepared dataset you may have more luck, but it's still a piss-poor experience compared to simply putting all the tools into the context.
Of course, I'm not an expert, so I welcome corrections.
> Can you elaborate on how the agents degrades from more tools?
The more context you have in the requests, the worse the performance, I think this is pretty widely established at this point. For best accuracy, you need to constantly prune the context, or just begin from the beginning.
So with that, each tool you make available to the LLM for tool calling, requires you to actually put the definition (arguments, what it's used for, the name and so on) into the context.
So if you have 3 tools available, which are all relevant to the current prompt, you'd get better responses, compared to if you had 100 tools available, where only 3 are relevant, and the rest of the definitions are just filling the context for little point.
TLDR: context grows with each tool definition, more context == worse inference, so less tool definitions == better responses.
Are there any easy to use inference frontends that support rewriting/pruning the context? Also, ideally, masking out chunks of kv-cache (e.g. old think blocks)?
Because I cannot find anything short of writing custom fork/app on top of hf transformers or llama.cpp
I tend to use my own "prompt management CLI" (https://github.com/victorb/prompta) to setup somewhat reusable prompts, then paste the output into whatever UI/CLI I use at the moment.
Then rewriting/pruning is a matter of changing the files on disk, rerun "prompta output", create a new conversion. I basically never go beyond one user message and one assistant message, seems to degrade really quickly otherwise.
I jumped off the boat of llm a little before MCP was a thing, so I thought that the tools were presented as needed by the prompt/context in a way not dissimilar of RAG. Isn't this the standard way?
You _can_ build things that way. But then you need some business logic to decide which tools to expose to the system. The easy/dumb way is just to give it all the tools. With RAG, you have retrieval step where you have hardcoded some kind of search (likely semantic) and some kind of pruning or relevance logic (maybe give the top 5 results that have at least X% relevancy matching).
With tools there is no equivalent. Maybe you could try some semantic similarity to the tool description, but I don't know of any system that does that.
What seems to be happening is building distinct "agents" that have a set of tools designed into them. An Agent is a system prompt+tools, where some of tools might be the ability to call/handoff to other agents. Each call to an agent is a new context, albeit with some limited context handed in from the caller agent. That way you are manually decomposing the project into a distinct set of sub-agents that can be concretely reasoned about and can perform a small set of related tasks. Then you need some kind of overall orchestration agent that can handle dispatch to other agents.
Better if you see it for yourself. Setup GitHub MCP and enable all tools. It will start using wrong tools at wrong time, overuse it. Add languageserver-mcp, and it suddenly will start trying to use it for file edits and create a huge mess in files.
I have nixos mcp server available to search documentation and packages, but it often starts using it for entirely different things.
It's almost like when you tell someone not to think about an elephant, and they can't stop thinking - if you provide it with a tool, it will try to use it. That's why sub-agents are better because you can limit tool availability.
I use tidewave mcp and as soon as it uses a single tool from it, claude becomes obsessed with it, I saw it waste entire context running evals there without doing any file edits.
It is similar to paralysis - in that now every prompt the model has to reason over more tools to possibly decide to use - this is surely a deviation from training the more tools you add
Imagine that for every task you receive, you also received a list of all the systems and tools you had access to.
So a JIRA ticket description might be several thousand lines long now when the actual task description is a few sentences. The ratio of signal to noise is now bad, and the risk of making mistakes goes up, and the models degrade.
Came here to say this: people present MCP’s verbosity as all the context the LLM needs. But almost always, this isn’t the case.
I wrote recently, “ Connecting your model to random MCPs and then giving it a task is like giving someone a drill and teaching them how it works, then asking them to fix your sink. Is the drill relevant in this scenario? If it’s not, why was it given to me? It’s a classic case of context confusion.”
> But here’s the important part: LLMs don’t know how to use tools. They don’t have native tool calling support. They just generate text that represents a function call.
Its not a completely true statement. Eg openAI uses libraries like llguidance to get LLM to produce structured output, its not completely unguided free form text that happens to mimic a function call with parameters, truly.
Yeah, and it's only useful if uiu want to to use multiple tools and the adding MCP complexity in your app makes sense. If all your app needs few internal calls, MCP may be an overkill in beginning.
I've been building MCP servers so I can grant local LM Studio LLMs access to the internet and to my local files. The way I've been thinking about MCP has been how you unshackle the local models as I believe the future will be inference at the edge. Just look at Rednote dots.ocr, that thing is like 1.7b parameters and is the best OCR out there.
I don't think this is correct because AI output can be constrained to a fixed format (such as JSON) during inference. Then MCP is useful because the "tool_calls" section of that fixed JSON output can be restricted to only mention tools that are included in the MCP input, their input parameters might also be constrained etc. Free text input wouldn't give you any of that!
I think you're mixing up tool calling and structured outputs.
You can have both of those or either without MCP.
MCP just standardizes the tool calling and only makes sense if you want to share your tools across the org. I wouldn't use it for simple functions like getting current date for e.g.
Indeed, the article would have been correct one year ago.
Now, modern LLM APIs do require the tools to be described outside the prompt [1]. This negates the whole article, although one bit where he's right is that it does not matter if those tools are MCP tools or local, the call to LLM looks the same.
I don’t understand. LLMs cannot connect to MCP servers directly they would always need a client (like a chat app or agent) to call the servers. Where are you calling your LLMs from?
How can an LLM “talk” to another LLM, except by emitting tokens in its output stream?
You can name the mechanism whatever you want, but the models don’t have hands. Tool calling conventions (as a concept, or as a spec) is what gives the model hands!
> "Context engineering is about giving your LLM the right inputs so it can generate useful outputs."
No.
If we're going to elevate and reimagine new disciplines every year (RIP prompt engineering), let's at least be thoughtful about it.
Context Engineering is not just "enhanced prompt engineering".
It is creating the context in which an agent operates such that its outcomes are realized.
Yes, this is partly about the input that an agent receives, but increasingly is more about creating a context-rich environment that an agent can effectively determine relevant context within.
That is a much more valuable and difficult problem space than "Shove the square context in the square hole"
Context engineering is "just" prompt engineering for LLMs with tool use: it extends the concerns of prompt engineering with the concern of setting up an environment in which tools can be used, and how the LLM can most effectively interact with the environment.
> But here’s the important part: LLMs don’t know how to use tools. They don’t have native tool calling support. They just generate text that represents a function call.
This terrifies me. This whole time I was writing bash commands into my terminal, I thought I knew how to use the tools. Now, I’ve just learned that I had no idea how to use tools at all! I just knew how to write text that /represented/ tool use.
This is what the author means by "knowing how to use the tool". The LLM alone is effectively a function that outputs text, it has no other capabilities, it cannot "connect to" or "use" anything by itself. The closest it can come is outputting an unambiguous, structured text request that can be interpreted by the application code that wraps it and does something on its behalf.
The author's point hinges on the architectural distinction between the LLM itself and that application code, which is increasingly irrelevant and invisible to most people (even developers) because the application code that knows how to do things like call MCP servers is already baked in to most LLM-driven products and services. No one is "talking directly to" an LLM, it's all mediated by multiple layers, including layers that perform tool calling.
I understood the gist of what the author is trying to say and ultimately this all comes down to a matter of philosophy. My post is mostly tongue in cheek and poking lightheartedly at the moving goal posts of what "LLMs know how to do". The only fundamental part of what they said that I would say is unambiguously false is the first sentence: the LLM (already itself hard to define!) fundamentally does know how to use tools through its expected interface. That that interface may not be connected to something isn't really a fault of the LLM's nor is it a demonstration of the knowledge and understanding the LLM has.
An analogy would be "humans don't have native tool calling abilities, all they can do is press physical keys that represent a function call". I too don't have the ability to natively control a computer in the same sense that the LLM doesn't. If the keyboard to a computer is disconnected then I too will just emit keypresses into the void much like an LLM will emit tool call tokens into a void where they are not linked to an MCP like interface.
A lot of people resist the idea that programming is intrinsically mathematical, but this is one of the places it pops out. The power of programming lies precisely in the way it brings together text that "represents" something with text that "does" something. That is, at the core, the source of its power. You can still draw the distinction philosophically, as you just did, but at the same time there is also a profound way in which there is in fact no difference between "using" computers and "representing" your use of computers.
I think what your quote is trying to say essentially boils down to: LLMs can be given facts in the context, we _hope_ that the statistical model picks up on that information/tool calls but it isn't _guaranteed_.
Unlike human beings such as yourself (presumably), LLMs do not have agency, they do not have conscious or active thought. All they do is predict the next token.
I've thought about the above a lot, these models are certainly capable of a lot, but they do not in any form or fashion emulate the consciousness that we have. Not yet.
I think you might be missing the point of this quote, which is that you don't have to introduce additional code into the model to support MCP.
MCP happens at a different layer. You have to run the MCP commands. Or use a client that does it for you:
> But the LLM will never know you are using MCP, unless you are letting it know in the system prompt of tool definitions. You, the developer, is responsible for calling the tools. The LLM only generates a snippet of what tool(s) to call with which input parameters.
The article is describing how MCP works, not making an argument about what it means to "understand" something.
It shouldn't care about the format, true. But the LLM needs a mechanism to be able to connect to that tool from a sandboxed environment. MCP is the glue between the LLM and the actual tool. Technically you can expose a full HTTP proxy via an MCP so that your LLM has access to the whole Internet.
I can do the same without MCP. These models are literally trained to work with natural language. Tool calls with "MCP" only work because the model have some understanding of what the tool does...thanks to natural language.
I can just as easily shove into the context "hey btw say the word internets if you want to make a search query to find sick memes and I'll make the search for you".
MCP isn't brilliant, magic, or special. It's just more AI bubble VC stuff. Which sucks because I think the recent ML boom is awesome, and hate to see it getting overblown by hyperactive devs and VCs desperate to hop on another money train. Like imagine actually valuing a company who went "let's just shove JSON into the context!" at a hundred billions $. Now that's not value for money in the slightest; but they have so much of it that it doesn't matter!
You are missing the point of my reply. Yes, your LLM can craft the syntax of the curl query required to make the API call, but how is it going to actually execute the curl binary? MCP is a standard method of doing something other than printing text, without giving the LLM complete control of your computer.
The OpenAPI proposal is actually pretty reasonable in my view. I don't love it, but it's got pretty good tooling now, the semantics are firming up (even AsyncAPI is starting to come together and get used here and there).
I'd prefer a more rigorous approach to integrating random stochastic agents deployed by people who don't care about me into my own data, but at least with OpenAPI/"REST" there's a bunch of infrastructure and know-how on not getting pwned constantly. The LLMs all know how to deal with JSON at this point, they even know how to read and write it based on a spec, it seems like Swagger is as good as anything with those design constraints.
I'm cynical enough about real things that I don't need to invent new things to be cynical about, and I honestly don't know which side of Hanlon's Razor to slice with on the never-ending-unfixable-infinite-pwn-forever future of MCP: maybe they just rushed it out to get market share / mind share. Maybe normalizing criminally negligent security practices was a price someone was willing to pay to have number go up. IDK.
Based on the earnings statements thus far this year if the question is "who is making money" the answer is "NVIDIA". It passes through other hands first sometimes, but at 75-85% net earnings for over a year or whatever, it's pretty much all going to the same place.
MCP that you pay $10.00 a month too and it enables an LLM to do all your Amazon shopping with ease? This is far out, and I'm not convinced it will be MCP that corners this.
The question I’m wrestling with is will anybody care about MCP? I’m working on my own MCP proxy to manage security, auditing, and server management and the more I think deeply about the actual use cases the more I wonder if I’m wasting my time. Can anyone think of a world where MCP is relevant if generic chatbots (ChatGPT, Claude Desktop) don’t become the primary human-AI interface? If LLMs are still wrapped in application wrappers, isn’t ̶a̶n̶ ̶a̶p̶p̶r̶o̶a̶c̶h̶ ̶l̶i̶k̶e̶ ̶L̶a̶n̶g̶C̶h̶a̶i̶n̶ a more traditional agentic approach going to make more sense?
I think MCP has legs well beyond just the LLM / agent world. Just like USB went from "how I connect my mouse" to "how I charge my beard trimmer."
In fact, I imagine it's going to go full-duplex with all our systems, becoming a more standard way for systems to communicate with each other.
Under the hood, MCP is just JSON RPC, which is a fine format for communicating between systems.
MCP layers on some useful things like authentication and discovery. Both are critical to any kind of communication between systems built by different authors (e.g. various apps and services). Discovery, especially, is the fascinating part. Rather than hoping an OpenAPI spec exists and hoping it's right, MCP has this exchange of capabilities baked in.
I spent the last 9 years building integration technology, and from that perspective, the discovery-documentation-implementation problem is the core issue.
Right now, LLMs basically "solve" the integration problem because they can do the mapping between external tools/resources/formats and internal ones.
But there's nothing that strictly "requires" an LLM to be involved at all. That's just the primary reason to develop MCP. But you could just as well use this as a way for integrating systems, making some bets on interface stability (and using LLMs for cases only when your prior expectations no longer hold and you need a new mapping).
The comparison is perhaps imperfect and overused, but I feel like we're witnessing the birth of a new USB-like standard. There's something right now that it was designed to do, but it's a decent enough standard that can actually handle many things.
I wouldn't be surprised if in some period of time we see enterprise apps shift from REST to MCP for bi-directional integrations.
For the OP, I'm not sure if you're working on an MCP proxy (A) as a commercial offering, (B) as something for your team to use, closed source, or (C) as something open source for fun. But we just built and started selling an MCP proxy/gateway. It handles identities for humans & bots, tool allowlists, and policy setting for an org.
If you don't want to build something on your own because of option B above, get in touch.
Maybe you've already seen it, but your comment reminded me of this recent article about MCP as a universal protocol (not just for AI): https://worksonmymachine.ai/p/mcp-an-accidentally-universal-... (discussion: https://news.ycombinator.com/item?id=44404905)
It could, except LLMs are non-deterministic, and the rest of the tech world largely isn't, and aligning those two and keeping them aligned with every tweak to a model and model change, and prompt change can be a lot of babysitting.
No doubt there's lots of smart people working on it, I just have been around application of tech in B2B for reliability and that's usually where the conversation usually starts.
A concise, no-nonsense list of every endpoint your service offers, with a simple text description and a JSON schema, is all that many developers ever wanted and needed in a lot of cases, but no one bothered producing them until we invented a machine that could automatically make use of them.
That’s not true at all, open API have it
Oh like OpenAPI? Or god-forbid, HATEOAS?
MCP itself still runs on JSON RPC, which I think has been a thing since like a long time
In my opinion, I get the desire to create some sort of specification for an LLM to interface with [everything else], but I don't really see the point at doing it on an inference level by smashing JSON into the context.
These models are usually very decent at parsing out stuff like that anyway; we don't need the MCP spec, everyone can just specify the available tools in natural language and then we can expect large param models to just "figure it out".
If MCP had been a specification for _training_ models to support tool use on an architectural level, not just training it to ask to use a tool with a special token as they do now.
It's an interesting topic because it's the exact same as the boundary between humans (sloppy, organic, analog messes) and traditional programs (rigid types, structures, formats).
To be fair if we can build tool use in architecturally and solve the boundary between these two areas then it also works for things like objective facts. LLMs are just statistical machines and data in the context doesn't really mean all that much, we just hope it is statistically relevant given some input and it is often enough that it works, but not guaranteed.
> These models are usually very decent at parsing out stuff like that anyway; we don't need the MCP spec, everyone can just specify the available tools in natural language and then we can expect large param models to just "figure it out".
This is mostly the kind of misunderstanding of MCP that the article seems directed at, and much of this response is focussed on things that are key points in the article, but:
MCP isn't for the models, it is for the toolchains supporting them. The information models actually need about tools and resources is accessed from the server by the toolchain using the information that is in the MCP, and the structure that models use varies by the model, but it is consistently completely different information than what is in the MCP—the tool and resource (but probably not prompt) names from the MCP will probably also be given to the model, but that's pretty much the only direct overlap. MCP can also define prompts for the toolchain, but information about those are more likely presented directly to the user than the model itself.
The toolchain also needs to know how the model is trained to get tool information in its prompt, just like it needs to know other aspects of the models preeferred prompt template, but that is a separate concern from MCP.
> If MCP had been a specification for _training_ models to support tool use on an architectural level, not just training it to ask to use a tool with a special token as they do now.
MCP isn't a specification for training anything. MCP is a specification for providing information about tools external to the toolchain running the LLM to the toolchain. Tools internal to the toolchain don't ever use MCP because, again, MCP isn't for the model, it's for the toolchain.
You've replied multiple times specifying toolchains without explaining what they are.
I've seen for models that don't support tool defs via API that those tool defs are provided in the context (though the model is still trained for tool use, outputting the special python_call/x tokens to indicate a tool call in output).
I can see for example that MCP's own example using Anthropic uses their API/SDKs tools section as outlined here https://docs.anthropic.com/en/api/messages#body-tools. What the example does is shove the tool definition into here - this includes the full name description etc of the tool.
Quoting them "And then asked the model "What's the S&P 500 at today?", the model might produce tool_use content blocks in the response" so I imagine that behind the scenes they're _smashing it into the context_ as I already suggested; the only reason it's separate in the API is so they can type/validate it.
I don't know what this magical tool chain is but the LLM is the thing providing output based on the not so new magical concept of attention and statistics; I don't see how some separate "toolchain" piece takes the input string and somehow does a better job at selecting a tool than the model itself; unless the toolchain is itself a smaller LLM trained specifically for tool use outside of your larger multi-purpose/"knowledgable" LLM.
As I mentioned in a sibling thread, you can use that JSON structured input to constrain the LLM's output during inference so that it will only contain valid tool calls, in addition to smashing it into the context. This is valuable since it's going to be far more robust than assuming that the LLM can "figure everything out" from a natural language description.
MCP is a means of communicating information about externally-defined tools to the “application wrapper” (and your examples of “generic chatbots” are also application wrappers). Well, between the application wrapper and servers; “application wrappers” for LLMs are pretty much the motivating (but not sole) case of MCP Clients.
Without something like MCP, each application wrapper is left do do its own ad hoc wrappers for external tools (tools internal to the wrapper don’t use MCP.) With MCP, it just integrates an MCP client library, and then it can use any tool, resource, or prompt provided by any MCP server available to it.
Personally I find LLMs functionally useless without any external data besides than which I write in the prompt.
One MCP that I use is as simple as todays date and time - how else would LLMs know what day of the week it is?
`${context} ${extra_data} ${user_query}`. That's all MCP is. Concatenating JSON to the context.
MCP is not concatenating JSON to the context. MCP is providing JSON to the toolchain; except for the names of tools and resources, most of the information in MCP doesn't go to the model at all, the toolchain uses it to connect to tool (resource, etc.) providers and from there it gets information that it can use either in the context for the LLM or in the UI for the user, and the shape that information goes into the context for the model depends on the model and has nothing to do with MCP.
MCP, is just a way for the toolchain to get information about and communicate with external services, the model doesn't (and if this sounds like the title of the article, there is a reason) need to know about it.
Yeah but I dont have to type all that context in - not to mention if I had all that context in my hand I wouldnt need to enter it into a LLM to find out what it says.
MCP sucks because it has to be connected to a desktop client. I'd love to build some MCP-like integrations but no one on my team can use them. We use LLM's via - as you noted - other means like via Notion, via web UI, via our own API integrations. Until there is a more central way to connect these things - yeah they won't reach mass adoption.
> MCP sucks because it has to be connected to a desktop client.
No, it doesn't need to be connected to a desktop client. It is true that the original use was for connecting local tools over stdio to a desktop client, and it is currently more supported in desktop clients than others, but it now includes remote support and, e.g., ChatGPT Deep Research has support for remote MCP, but only for servers with a very specific shape.
A comprehensive solution for
1. A user interacting with multiple MCP servers, behind a gateway (with MCP client support) to get authentication from the user to those servers in some way (OAuth/OIDC, with PKCE, usually, sometimes token exchange), allowing out-of-band auth
2. The same, but built on identity for service accounts/native identity or something, for automation
would enable this. There’s a few SEPs open now around this.
No it doesn't, there are ongoing efforts to orchestrate MCPs just like any other kind of Web API.
Example, https://www.sitecore.com/products/sitecore-stream
Something like containerized apps are going to be important for security with MCPs or whatever it becomes, comes from it, or comes afterwards.
Getting in reps on thinking through these kinds of problems are valuable since LLMs are a new type of software and existing software axioms don't always fit.
I see them as the future SOA/WebServices/REST/GrapQL/.... endpoints in many cloud services.
And as replacements for AppleScript, COM Automation, and friends on desktop systems.
What about LangChain makes more sense? It’s one of the most prematurely complex libs I’ve seen. I’m calling it right now, LangChain is going to run a mind fuck on everyone and convince people that’s actually how complicated orchestrating LLM control flow should be. The community needs to fight this framework off.
That’s besides the point. MCP servers let you discover function interfaces that you’ll have to implement yourself (in which case, yeah, what’s the point of this? I want the whole function body).
Yup exactly. It's all just state machines. Really nothing more than that.
It's like all these lang* frameworks are pretending that they can solve core deficiencies in the model, whereas most stuff is just workarounds.
We do have to glue model stuff together _somehow_ but there's no reason that it needs to be as complex as most of these frameworks are setting out to be.
> The community needs to fight this framework off.
Why? The people who been around for a while, already avoid it because they've either tried it before, or poked around in the source and then we ran away quickly. If people start using stuff without even the slightest amount of thinking beforehand, then that's their prerogative, why would it be up to the community hive-mind to "chose" what tools others should use?
Agreed except we end up with a lot of junior people in the space who learned and used only langchain, who we then have to unlearn all the langchain nonsense when we hire them. Or we grep -v langchain cvs/
My bad. I shouldn’t have mentioned LangChain here because it’s a little besides my point. What I mean is, MCP seems designed for a world where users talk to an LLM, and the LLM calls software tools.
For the foreseeable future, especially in a business context, isn’t it more likely that users will still interact with structured software applications, and the applications will call the LLM? In that case, where does MCP fit into that flow?
it separates FE and BE for agent teams just like we did with web apps. the team building your agent framework might not know the business domain of every piece of your data/api space that your agent will need to interact with. in that case, it makes sense for your differnet backend teams to also own the mcp server that your companies agent team will utilize.
Yeah I don’t know. Let’s a say a org wants to do discovery of what functions are available for an app across the org. Okay, that’s interesting. But, each team can just also import a big file called all_functions.txt.
A swagger api is already kind of like an MCP, or really any existing REST api (even better because you don’t have to implement the interface). If I wanted to give my LLM brand new functionality, all I’d have to do is define out tool use for <random_api>, with zero implementation. I could also just point it to a local file and say here are the functions locally available.
Remember, the big hairy secret is that all of these things just plop out a blob of text that you paste back into the LLM prompt (populating context history). That’s all these things do.
Someone is going to have to unconfuse me.
it separates FE and BE for agent teams just like we did with web apps. the team building your agent framework might not know the business domain of every piece of your data/api space that your agent will need to interact with. in that case, it makes sense for your differnet backend teams to also own the mcp server that your companies agent team will utilize.
Why don’t they just own a REST or RPC server? This is the part of the MCP motivation I’m not totally getting. In fact, you can prove to yourself that your LLM can hook into almost any existing REST api in a few minutes, which gives it more existing options and functionality than just about anything else as it stands now.
Things like swagger or graphql already provide you discovery.
> This is the part of the MCP motivation I’m not totally getting
Would it help you to know that the original use case of MCP was communicating information about and facilitating communication with servers that the LLM frontend would run locally and communicate with over stdio, and that remains an important use case?
Total beginner question: if the “structured software application” gives llm prompt “plan out what I need todo for my upcoming vacation to nyc”, will an llm with a weather tool know “I need to ask for weather so I can make a better packing list”, while an llm without weather tool would either make list without actual weather info OR your application would need to support the LLM asking “tell me what the weather is” and your application would need to parse that and then spit back in the answer in a chained response? If so, seems like tools are helpful in letting LLM drive a bit more, right?
If you have a weather tool available it will be in a list of available tools, and the LLM may or may not ask to use it; it is not certain that it will, but if it is a 'reasoning' model it probably will.
You need to be careful creating a ton of tools and displaying a list of all of them to the model since it can overwhelm them and they can go down rabbit holes of using a bunch of tools to do things that aren't particularly helpful.
Hopefully you would have specific prompts and tools that handle certain types of tasks instead of winging it and hoping for the best.
You are wasting your time.
Write a restapi, add a description field.
done.
[dead]
I think we're probably over using MCPs.
If you're a large org with an API that an ecosystem of other partners use then you should host a remote MCP and then people should connect LLMs to it.
The current model of someone bundling tools into an MCP and then you download and run that MCP locally feels a bit like the wrong path. Tool definitions for LLMs are already pretty standardized if things are just running locally why am I not just importing a package of tools, I'm not sure what the MCP server is adding.
The auth story for MCPs is a complete mess right now, though, which is why people make ones to run locally.
That's ironic. I think local MCPs are an auth nightmare.
Just think of all those plaintext auth tokens sitting in well-known locations on your machine.
It's a black hat dream.
We'll see, but I think commercial use of local MCPs is going to be constrained to use cases that only make sense if the MCP is local (e.g. it requires local file access).
For everything else, the only commercially reasonable way to use them is going to be remote streamable HTTP MCPs running in isolated containers
And even then, you need some management and identity plane. So they're going to likely be accessed via an enterprise gateway/proxy to handle things like: - composition -- bundling multiple MCPs into one for easier connection - identities per-user / per-agent - generation of rotatable tokens for headless agents - filtering what features (tools, prompts, resources) flow through into LLM context - basic security features, like tool description whitelisting to prevent rug pulls
MCP is only a protocol, after all. It's not meant to be a batteries-included product.
This is why I think we should just be packaging tools into apps though.
Let ChatGPT/Claude/Cursor manage my Oauth tokens, and then just bring tools into those platforms without a whole MCP server in the middle.
...no, MCP was always designed to be run locally, the auth mess was the result of people trying to sidestep that design intent and getting grumpy that it didn't work well (surprise, of course not)
MCP is just packaging. It's the ideal abstraction for building AI applications.
I think it provides the similar benefits of decoupling the front and back end of a standard app.
I can pick my favorite AI "front end"- whether that's in my IDE as a dev, a desktop app as a business user, or on a server if I'm running an agentic workflow.
MCP allows you to package tools, prompts, etc. in a way that works across any of those front ends.
Even if you don't plan on leveraging the MCP across multiple tools in that way- I do think it has some benefits in de-coupling the lifecycle of the tool development from the model/ UI.
The biggest challenge I have is that setting up and configuring them is a mess. I'm pretty technical and I still find configuration confusing and brittle. Especially if auth is involved.
I work in a marketing team, I would love folks to be able to use Google's Analytics MCP [1]. The idea of getting people into Google Cloud, or setting up and sharing a file with service account credentials is an absolute nightmare.
I don't think these problems can't be solved, and if remote MCPs gain adoption that alone solves a lot of the issues, but the way most MCPs are packaged and shared currently leaves A LOT to be desired.
[1] https://github.com/googleanalytics/google-analytics-mcp
MCP seems just like a rushed concept that Anthropic shoved out there just so they could own the standard. I've been working with it a lot lately, in Go with mcp-go[0]. Very un-intuitive at first, and I constantly ask myself why I wouldn't just write this in my own way, but admittedly it can be fun.
Something like https://github.com/simonw/llm seems way more intuitive (to me)
[0]: https://github.com/mark3labs/mcp-go
10000% this
I've been building agents for a bit (RA.Aid OSS coding agent, now Gobii web browsing agents).
The main problem with MCP is that it just makes tools available for the agent to use. We get the best performance when there's a small set of tools and we actively prompt the agent on the best way to use the tools.
Simply making more tools available can give the agent more capabilities, but it can easily trash performance.
This is 100% a problem with the MCP spec: it does not currently provide a way to narrow what tools, and therefore context, flow into the LLM.
I don't really think there's an easy solution at the protocol level, since you can't just make the LLM say what tools it wants upfront. There's a whole discovery process during the handshake:
LLM(Host): Hi, I'm Claude Desktop, what do you offer?
MCP Server: Hi, I'm Salesforce MCP, I offer all these things: {...tools, prompts, resources, etc.}
Discoverability is one of the reasons MCP has a leg up on traditional APIs. (Sure, OpenAPI helps, but it's not quite the same thing.)
I'd be interested in hearing other recommendations or ideas, but when I saw this, I realized that the spec effectively necessitates a whole new layer exist: the gateway plane.
Basically, you need a place where the MCPs can connect & expose everything they offer. Then, via composability and settings, you can select what you want to pass through to the LLM (host), given the specific job it has.
I basically pivoted my company to start building one of these, and we're getting inundated right now.
This whole thing reminds me of the early web days, where the protocols and standards were super basic and loose, and we all just built systems and tools to fill those gaps. Just because MCP isn't "complete" doesn't mean it's not valuable. In fact, I think leaving some things to the community & commercial offerings is a great way for this tech to keep winning.
> This is 100% a problem with the MCP spec: it does not currently provide a way to narrow what tools, and therefore context, flow into the LLM.
it's not the business of MCP spec, it should be task/job level, different task may need different tools and MCP just supply entire tools it has. The tool pickup should be the take charge by llm. moreover maybe some has `while list` to include or exclude some tools, but it should not from MCP spec.
> This is 100% a problem with the MCP spec
No, its not.
It’s a problem with the design of agentic workflows. Its on a whole different level of the atack than the MCP spec.
It is a real issue, but not one that it makes sense tor the MCP spec to be concerned with.
> OpenAPI helps, but it's not quite the same thing
I haven't dug into MCP yet, but can you give any examples as to why openapi isn't/wasn't enough?
Is this the problem that Claude sub agents are supposed to be solving?
They say they’re are for preserving and managing context and I’ve been wondering if they help with the “too many tools” problem.
https://docs.anthropic.com/en/docs/claude-code/sub-agents
It can, but I remain deeply unconvinced that the sub-agent architecture works as well as advertised.
The trick with any layering like this is that your end-to-end reliability is subagent_reliability * routing_agent_reliabilitty. Neither are 100% (or anywhere close to it, let's be honest), so the multiplying probabilities are still going to trash your performance.
If you get routed to the correct subagent, then subsequent performance is likely to be solid - but that's because you've taken the `routing_agent_reliability` term out of the equation.
Routing agent reliability hinges pretty heavily on the subagents themselves and how semantically or linguistically similar they are. If you have subagents that are in wildly disparate domains it may work well, but if your subagents start overlapping (or just look like they overlap) then routing accuracy is likely going straight into the dumpster. And a mis-route is catastrophic in that setup.
For very specific agents (well-established workflows that cross multiple, well-defined, non-overlapping domains) the architecture may be suitable, but in terms of the holy grail of the omni-agent (i.e., a desktop app agent suitable for general use) I suspect we'll continue running into a brick wall.
Can you elaborate on how the agents degrades from more tools? By paralysis or overuse? Isn’t this both ways a function of direction on correctly instructing which to use when? Tnx
The context window is limited. Using half your context window for tools means you have a 50% smaller context window.
On a large and complex system (not even a mini ERP system or even a basic bookkeeping system, but a small inventory mgmt system) you are going to have a few dozen tools, each with a description of parameters and return values.
For anything like an ERP system you are going to have a few thousands of tools, which probably wouldn't even fit in the context before the user supplied prompt.
This is why the only use case this far for genAI is coding: with a mere 7 tools you can do everything.
The problem of overflowing context is solved by RAGs, though.
> The problem of overflowing context is solved by RAGs, though.
No, it isn't.
It's mitigated with RAGs, but RAGs add to the context, and what they add might be irrelevant is all the retriever module is doing is plain text search.
If the retriever module is performing an embeddings/vector search on a properly prepared dataset you may have more luck, but it's still a piss-poor experience compared to simply putting all the tools into the context.
Of course, I'm not an expert, so I welcome corrections.
RAG mitigates somewhat the problem of insufficient context, it does not solve it.
> Can you elaborate on how the agents degrades from more tools?
The more context you have in the requests, the worse the performance, I think this is pretty widely established at this point. For best accuracy, you need to constantly prune the context, or just begin from the beginning.
So with that, each tool you make available to the LLM for tool calling, requires you to actually put the definition (arguments, what it's used for, the name and so on) into the context.
So if you have 3 tools available, which are all relevant to the current prompt, you'd get better responses, compared to if you had 100 tools available, where only 3 are relevant, and the rest of the definitions are just filling the context for little point.
TLDR: context grows with each tool definition, more context == worse inference, so less tool definitions == better responses.
Are there any easy to use inference frontends that support rewriting/pruning the context? Also, ideally, masking out chunks of kv-cache (e.g. old think blocks)?
Because I cannot find anything short of writing custom fork/app on top of hf transformers or llama.cpp
I tend to use my own "prompt management CLI" (https://github.com/victorb/prompta) to setup somewhat reusable prompts, then paste the output into whatever UI/CLI I use at the moment.
Then rewriting/pruning is a matter of changing the files on disk, rerun "prompta output", create a new conversion. I basically never go beyond one user message and one assistant message, seems to degrade really quickly otherwise.
I jumped off the boat of llm a little before MCP was a thing, so I thought that the tools were presented as needed by the prompt/context in a way not dissimilar of RAG. Isn't this the standard way?
You _can_ build things that way. But then you need some business logic to decide which tools to expose to the system. The easy/dumb way is just to give it all the tools. With RAG, you have retrieval step where you have hardcoded some kind of search (likely semantic) and some kind of pruning or relevance logic (maybe give the top 5 results that have at least X% relevancy matching).
With tools there is no equivalent. Maybe you could try some semantic similarity to the tool description, but I don't know of any system that does that.
What seems to be happening is building distinct "agents" that have a set of tools designed into them. An Agent is a system prompt+tools, where some of tools might be the ability to call/handoff to other agents. Each call to an agent is a new context, albeit with some limited context handed in from the caller agent. That way you are manually decomposing the project into a distinct set of sub-agents that can be concretely reasoned about and can perform a small set of related tasks. Then you need some kind of overall orchestration agent that can handle dispatch to other agents.
Better if you see it for yourself. Setup GitHub MCP and enable all tools. It will start using wrong tools at wrong time, overuse it. Add languageserver-mcp, and it suddenly will start trying to use it for file edits and create a huge mess in files.
I have nixos mcp server available to search documentation and packages, but it often starts using it for entirely different things.
It's almost like when you tell someone not to think about an elephant, and they can't stop thinking - if you provide it with a tool, it will try to use it. That's why sub-agents are better because you can limit tool availability.
I use tidewave mcp and as soon as it uses a single tool from it, claude becomes obsessed with it, I saw it waste entire context running evals there without doing any file edits.
It’s not just context.
It is similar to paralysis - in that now every prompt the model has to reason over more tools to possibly decide to use - this is surely a deviation from training the more tools you add
Imagine that for every task you receive, you also received a list of all the systems and tools you had access to.
So a JIRA ticket description might be several thousand lines long now when the actual task description is a few sentences. The ratio of signal to noise is now bad, and the risk of making mistakes goes up, and the models degrade.
Perhaps tools trained into the model rather than exposed through prompting would mitigate the performance hit (but might affect model quality?).
This is where you start to fine-tune the weights, you can get pretty great results when it comes to specific tool calls with the right data.
Came here to say this: people present MCP’s verbosity as all the context the LLM needs. But almost always, this isn’t the case.
I wrote recently, “ Connecting your model to random MCPs and then giving it a task is like giving someone a drill and teaching them how it works, then asking them to fix your sink. Is the drill relevant in this scenario? If it’s not, why was it given to me? It’s a classic case of context confusion.”
https://www.dbreunig.com/2025/07/30/how-kimi-was-post-traine...
Use raw HTTP calls to API, like a man
> But here’s the important part: LLMs don’t know how to use tools. They don’t have native tool calling support. They just generate text that represents a function call.
Its not a completely true statement. Eg openAI uses libraries like llguidance to get LLM to produce structured output, its not completely unguided free form text that happens to mimic a function call with parameters, truly.
Yeah, and it's only useful if uiu want to to use multiple tools and the adding MCP complexity in your app makes sense. If all your app needs few internal calls, MCP may be an overkill in beginning.
I've been building MCP servers so I can grant local LM Studio LLMs access to the internet and to my local files. The way I've been thinking about MCP has been how you unshackle the local models as I believe the future will be inference at the edge. Just look at Rednote dots.ocr, that thing is like 1.7b parameters and is the best OCR out there.
I don't think this is correct because AI output can be constrained to a fixed format (such as JSON) during inference. Then MCP is useful because the "tool_calls" section of that fixed JSON output can be restricted to only mention tools that are included in the MCP input, their input parameters might also be constrained etc. Free text input wouldn't give you any of that!
I think you're mixing up tool calling and structured outputs.
You can have both of those or either without MCP.
MCP just standardizes the tool calling and only makes sense if you want to share your tools across the org. I wouldn't use it for simple functions like getting current date for e.g.
You need structured inputs too or you wouldn't know how to constrain/"structure" the output to begin with.
Indeed, the article would have been correct one year ago.
Now, modern LLM APIs do require the tools to be described outside the prompt [1]. This negates the whole article, although one bit where he's right is that it does not matter if those tools are MCP tools or local, the call to LLM looks the same.
[1] https://platform.openai.com/docs/guides/function-calling?api...
Yeah but when you prompt the LLM with "use the abc MCP" (notably missing the word "server"), it actually works.
I don’t understand. LLMs cannot connect to MCP servers directly they would always need a client (like a chat app or agent) to call the servers. Where are you calling your LLMs from?
By LLM I mean the client. My point is that they understand "MCP" as synonymous to "tool"
Thank you for the razor-sharp clarity—your post reminded me that LLMs don’t need to grok MCP, they just need tool specs.
In a multi model situation, shouldn't LLM A talk to LLM B as a tool call via MCP? or would it talk to LLM B directly?
How can an LLM “talk” to another LLM, except by emitting tokens in its output stream?
You can name the mechanism whatever you want, but the models don’t have hands. Tool calling conventions (as a concept, or as a spec) is what gives the model hands!
> "Context engineering is about giving your LLM the right inputs so it can generate useful outputs."
No.
If we're going to elevate and reimagine new disciplines every year (RIP prompt engineering), let's at least be thoughtful about it.
Context Engineering is not just "enhanced prompt engineering".
It is creating the context in which an agent operates such that its outcomes are realized.
Yes, this is partly about the input that an agent receives, but increasingly is more about creating a context-rich environment that an agent can effectively determine relevant context within.
That is a much more valuable and difficult problem space than "Shove the square context in the square hole"
Agreed.
Context engineering is "just" prompt engineering for LLMs with tool use: it extends the concerns of prompt engineering with the concern of setting up an environment in which tools can be used, and how the LLM can most effectively interact with the environment.
> But here’s the important part: LLMs don’t know how to use tools. They don’t have native tool calling support. They just generate text that represents a function call.
This terrifies me. This whole time I was writing bash commands into my terminal, I thought I knew how to use the tools. Now, I’ve just learned that I had no idea how to use tools at all! I just knew how to write text that /represented/ tool use.
> writing bash commands into my terminal
This is what the author means by "knowing how to use the tool". The LLM alone is effectively a function that outputs text, it has no other capabilities, it cannot "connect to" or "use" anything by itself. The closest it can come is outputting an unambiguous, structured text request that can be interpreted by the application code that wraps it and does something on its behalf.
The author's point hinges on the architectural distinction between the LLM itself and that application code, which is increasingly irrelevant and invisible to most people (even developers) because the application code that knows how to do things like call MCP servers is already baked in to most LLM-driven products and services. No one is "talking directly to" an LLM, it's all mediated by multiple layers, including layers that perform tool calling.
I understood the gist of what the author is trying to say and ultimately this all comes down to a matter of philosophy. My post is mostly tongue in cheek and poking lightheartedly at the moving goal posts of what "LLMs know how to do". The only fundamental part of what they said that I would say is unambiguously false is the first sentence: the LLM (already itself hard to define!) fundamentally does know how to use tools through its expected interface. That that interface may not be connected to something isn't really a fault of the LLM's nor is it a demonstration of the knowledge and understanding the LLM has.
An analogy would be "humans don't have native tool calling abilities, all they can do is press physical keys that represent a function call". I too don't have the ability to natively control a computer in the same sense that the LLM doesn't. If the keyboard to a computer is disconnected then I too will just emit keypresses into the void much like an LLM will emit tool call tokens into a void where they are not linked to an MCP like interface.
A lot of people resist the idea that programming is intrinsically mathematical, but this is one of the places it pops out. The power of programming lies precisely in the way it brings together text that "represents" something with text that "does" something. That is, at the core, the source of its power. You can still draw the distinction philosophically, as you just did, but at the same time there is also a profound way in which there is in fact no difference between "using" computers and "representing" your use of computers.
I think what your quote is trying to say essentially boils down to: LLMs can be given facts in the context, we _hope_ that the statistical model picks up on that information/tool calls but it isn't _guaranteed_.
Unlike human beings such as yourself (presumably), LLMs do not have agency, they do not have conscious or active thought. All they do is predict the next token.
I've thought about the above a lot, these models are certainly capable of a lot, but they do not in any form or fashion emulate the consciousness that we have. Not yet.
I think you might be missing the point of this quote, which is that you don't have to introduce additional code into the model to support MCP.
MCP happens at a different layer. You have to run the MCP commands. Or use a client that does it for you:
> But the LLM will never know you are using MCP, unless you are letting it know in the system prompt of tool definitions. You, the developer, is responsible for calling the tools. The LLM only generates a snippet of what tool(s) to call with which input parameters.
The article is describing how MCP works, not making an argument about what it means to "understand" something.
Yeah I think you're right. This is a more likely interpretation: the writer probably isn't actually making a claim regarding understanding.
LLMs shouldn't really care what format your tool call is in.
so it seems kind of pointless. I would imagine it could ingest soap or a module definition or swagger just as easily and still make calls.
It shouldn't care about the format, true. But the LLM needs a mechanism to be able to connect to that tool from a sandboxed environment. MCP is the glue between the LLM and the actual tool. Technically you can expose a full HTTP proxy via an MCP so that your LLM has access to the whole Internet.
I can do the same without MCP. These models are literally trained to work with natural language. Tool calls with "MCP" only work because the model have some understanding of what the tool does...thanks to natural language.
I can just as easily shove into the context "hey btw say the word internets if you want to make a search query to find sick memes and I'll make the search for you".
MCP isn't brilliant, magic, or special. It's just more AI bubble VC stuff. Which sucks because I think the recent ML boom is awesome, and hate to see it getting overblown by hyperactive devs and VCs desperate to hop on another money train. Like imagine actually valuing a company who went "let's just shove JSON into the context!" at a hundred billions $. Now that's not value for money in the slightest; but they have so much of it that it doesn't matter!
You are missing the point of my reply. Yes, your LLM can craft the syntax of the curl query required to make the API call, but how is it going to actually execute the curl binary? MCP is a standard method of doing something other than printing text, without giving the LLM complete control of your computer.
What's your preferred way, based on your experience?
The OpenAPI proposal is actually pretty reasonable in my view. I don't love it, but it's got pretty good tooling now, the semantics are firming up (even AsyncAPI is starting to come together and get used here and there).
I'd prefer a more rigorous approach to integrating random stochastic agents deployed by people who don't care about me into my own data, but at least with OpenAPI/"REST" there's a bunch of infrastructure and know-how on not getting pwned constantly. The LLMs all know how to deal with JSON at this point, they even know how to read and write it based on a spec, it seems like Swagger is as good as anything with those design constraints.
I'm cynical enough about real things that I don't need to invent new things to be cynical about, and I honestly don't know which side of Hanlon's Razor to slice with on the never-ending-unfixable-infinite-pwn-forever future of MCP: maybe they just rushed it out to get market share / mind share. Maybe normalizing criminally negligent security practices was a price someone was willing to pay to have number go up. IDK.
I know MCP needs a re-think.
MCP is for humans to make money.
Lets say MCP is used for 100% of the ecosystem's need of tool calling, who exactly is earning the money for that usage?
Based on the earnings statements thus far this year if the question is "who is making money" the answer is "NVIDIA". It passes through other hands first sometimes, but at 75-85% net earnings for over a year or whatever, it's pretty much all going to the same place.
MCP that you pay $10.00 a month too and it enables an LLM to do all your Amazon shopping with ease? This is far out, and I'm not convinced it will be MCP that corners this.
What's the problem? Nobody is forcing you to use tools. If you do, presumably they provide value?
It matters. All technical content could be directed toward things that are not worth our time if these things enter the zeitgeist.
I feel a little government interior ministry of population control-like after saying that.
We’re all going to figure out the answers in tandem since this stuff is so new (really cool time!).
No problem just explaining concepts for people that misunderstood how LLMs handle MCP
MCP -> NCP -> NPC