How to Configure MCP, Function Calling, and API Gateway: A Three-Layer Architecture for AI Agent Tool Integration

Many people ask a common question when implementing tool calling for AI Agents:

I already have Function Calling. Do I still need MCP?

This question sounds reasonable, but it actually conflates different architectural layers.

Function Calling, MCP, and API Gateway are not three mutually exclusive alternatives. They form a three-layer structure from the model to the production system: Function Calling enables the model to express “which tool I want to call and what the parameters are”; MCP standardizes the integration of external tools, files, databases, SaaS services, and contextual resources; and the API Gateway manages identity verification, rate limiting, auditing, isolation, and cost control in real-world production environments.

If you are only building a demo, you might only need Function Calling. However, if you want an Agent to simultaneously connect to file systems, GitHub, databases, internal knowledge bases, browsers, and multiple business tools, you will start to need MCP. If this Agent needs to serve real users, access real systems, and perform actual operations, having MCP alone is insufficient—you also need a gateway, permissions, auditing, and degradation strategies in front of it.

This article will no longer repeat the discussion on “MCP vs. Function Calling: which is better?” I have already covered that separately: MCP vs Function Calling: Differences, Use Cases, and When to Use MCP?. Today, we focus on a more engineering-oriented question: How should the tool integration layer be structured for an AI Agent that is truly ready for production deployment?

1. Function Calling is the Invocation Expression Layer

The primary problem Function Calling solves is enabling the model to express invocation intent in a structured format.

For example, a user asks: “Help me check the order status for this user over the last 30 days.” The model cannot directly access the business system. You can provide it with a tool definition, specifying the tool’s name, required parameters, and parameter types. Upon seeing the tool definition, the model does not query the data itself; instead, it returns a structured invocation request. Your application code then extracts the parameters, executes the actual query, and returns the results to the model to continue generating the answer.

Therefore, the core value of Function Calling lies in structuring tool parameters, reducing errors in natural language parsing, allowing the model to choose among multiple tools, retaining final execution authority for business code, and enabling the invocation process to enter logging and debugging pipelines.

However, it has clear boundaries.

Function Calling does not automatically manage a fleet of tool services, discover available resources in a file system, handle user authorization, or natively solve issues related to multi-client, multi-tool, or multi-Agent reuse. It acts more like an “invocation contract” between the model and the application: the model intends to call a specific tool and provides the necessary parameters.

Issues such as where the tool resides, who has permission to invoke it, how to handle failures, how to log invocations, and whether write operations are allowed are not fully addressed by Function Calling itself.

2. MCP is the Protocol Layer for Tools and Context

The value of MCP is not to replace Function Calling. Instead, it addresses a different layer of problems: how to integrate an increasing number of tools, resources, and contexts into the AI application ecosystem using a unified protocol.

MCP involves several key roles: the Host is the LLM application initiating the connection (e.g., IDEs, chat applications, Agent platforms); the Client is the connector within the Host; and the Server is the service providing tool, resource, and contextual capabilities.

With standard Function Calling, you might end up rewriting tool definitions for every application: one set of GitHub tools for the IDE, another for the chat application, yet another for automated workflows, and a fourth for the internal Agent platform. This quickly becomes unmanageable.

MCP’s approach is to package capabilities such as GitHub, file systems, databases, browsers, and knowledge bases into MCP Servers. Different Hosts then connect to these servers via MCP Clients.

In other words, MCP functions more as the tool protocol layer of the AI era. It transforms tools from hardcoded functions within a single application into reusable context services that can be leveraged by multiple AI clients.

An MCP Server exposes three types of capabilities: Resources provide context and data for the model; Prompts offer reusable prompt templates and workflows; and Tools represent function capabilities that the model can request to execute.

This approach is better suited for complex systems than simple Function Calling alone.

For example, suppose you are building an internal R&D Agent that needs to read code repositories, query issues, access design documents, retrieve database changes, execute restricted tasks, and write results back to the project management system. While you could implement all these tools using Function Calling, maintenance costs would quickly escalate. Tool definitions would be scattered across applications, and permission boundaries would become entangled with business logic. Furthermore, switching Hosts would require re-implementing the entire integration.

MCP is better suited for standardizing, service-orienting, and protocolizing these external capabilities. This is why I repeatedly emphasize in MCP Server Production Governance and MCP Security Best Practices that the core value of MCP is not “adding another invocation method,” but rather “establishing a distinct boundary for tools and context.”

3. API Gateway as the Production Governance Layer

But we’re not done yet.

Many assume that once MCP is in place, Agents can directly connect to internal systems. This is actually unstable.

Because an MCP Server may still connect to real files, databases, APIs, and business systems. Whenever real systems are involved, the problem shifts from “how to invoke tools” to “how to manage boundaries.”

This is where an API Gateway comes in.

An API Gateway does not make the model smarter, nor does it replace MCP. Instead, it handles the most granular and often overlooked aspects of production environments: user identity, tenant isolation, tool invocation whitelisting, request signing, rate limiting, logging and auditing, cost control, field filtering, write-operation confirmation, and failure degradation.

Consider this example. You have a financial report analysis Agent that reads annual reports uploaded by users, extracting KPIs, risk factors, source pages, and evidence. While this scenario appears to be merely “reading files,” production requirements include at least: file ownership, task status, evidence page numbers, failure retries, manual review, and result boundaries. Especially with financial content, the system must prevent outputs containing buy/sell recommendations, target prices, or position advice.

MCP cannot fully cover these concerns on its own.

MCP can expose “PDF parsing tools,” “table extraction tools,” and “JSON Schema validation tools”; Function Calling allows the model to express which tool to invoke. However, permissions, auditing, user isolation, write restrictions, and risk warnings still require governance at the gateway and business layers.

Therefore, I prefer to split AI Agent tool integration into three layers:

LLM
  ↓ Function Calling
Agent Runtime
  ↓ MCP Client
MCP Server
  ↓ API Gateway
业务 API / 数据库 / 文件 / SaaS / 内部系统

Function Calling clarifies what the model needs. MCP brings tools and context in via a standard protocol. API Gateway keeps these calls controlled, auditable, and rollable back in production.

You can drop one layer and still run a demo, but once you go live, that missing layer becomes an entry point for incidents.

4. How do the three layers divide responsibilities?

Layer	Primary Responsibility	Out of Scope	Suitable Scenarios
Function Calling	Structured expression of tool-calling intent	Tool discovery, resource management, permission governance, auditing	Few tools, backend-controlled, demos or small-scale business logic
MCP	Standardized integration of tools, resources, prompts, and context	Full business-side authentication, rate limiting, billing, compliance policies	Multiple tools, multiple clients, multi-agent reuse
API Gateway	Authentication, rate limiting, auditing, tenant isolation, cost control, risk management	Model inference, tool protocols, context understanding	Real production environments, user data, write operations, enterprise systems

The key takeaway from this table is: don’t force protocol governance into Function Calling, and don’t expect MCP to handle full production governance for you.

Many projects run into trouble because they mix all three layers together.

For example, cramming dozens of tools into a single Function Calling request; making each tool’s description so long that the model’s selection becomes increasingly unstable; having the MCP Server connect directly to the business database without gateway isolation; lacking manual confirmation for write operations; no fallback when tools fail; only logging the model’s final answer while ignoring tool_call and tool_output; failing to distinguish between read and write permissions; and having no cost caps.

These issues aren’t obvious at first. During the demo phase, if the Agent can answer questions, it feels like the system is ready to go. But in production, the real problems aren’t “can it call tools?” but rather “what happens if it calls the wrong tool?”, “what if it’s too slow?”, “what if call costs spiral out of control?”, “what if users exceed their permissions?”, “what if tools return dirty data?”, and “what if the model treats tool descriptions as factual truth?”.

None of these issues can be solved by a single tools: [] parameter.

5. When should you use only Function Calling?

Using only Function Calling is fine, provided your system is simple enough.

I use Function Calling directly in these scenarios: very few tools; no need for cross-client reuse; low business risk; still validating product direction.

For instance, if you only have tools like get_user_profile, get_order_status, and create_support_ticket, all provided by the same backend service, with clear parameter structures and permission logic already handled by your existing business system, then there’s no need to introduce MCP right away.

Early MVPs are most at risk of being bogged down by “architectural correctness.” It’s more realistic to get the user path working with Function Calling first, then decide whether to extract an MCP Server later.

However, even when using Function Calling, I recommend doing three things: keep schemas as strict as possible, ensure both tool_call and tool_output are logged, and add separate confirmation steps for write operations—don’t mix them with read-only tools.

This stage can be simple, but it cannot lack boundaries entirely.

6. When should you introduce MCP?

When you start seeing these signals, it’s time to consider MCP: the same tool needs to be reused across multiple AI clients; the number of tools is growing, making maintenance of tools schemas painful; Agents need to integrate diverse resources like files, databases, GitHub, browsers, and knowledge bases; you want to decouple tool capabilities from application code; or you want different hosts to share the same context services.

For example, consider a solo developer’s website operations Agent. It might need to integrate Search Console data, GA4 page data, Cloudflare 404 logs, Astro local content inventory, GitHub repositories, article publishing systems, UTM Builders, and SEO inventory reports.

If you cram everything into a single Function Calling list, it quickly becomes a chaotic mishmash of tools.

A more sensible approach is to split them across MCP Servers:

xbstack-content-mcp
xbstack-analytics-mcp
xbstack-github-mcp
xbstack-publisher-mcp

Each Server manages a specific category of resources. The Agent Runtime is solely responsible for connecting to the appropriate MCP Server based on the task.

This way, whether you use the Chat interface, CLI, backend operations console, or IDE plugin, you can all reuse the same set of tool capabilities. This is the true value of MCP.

7. When is an API Gateway Mandatory?

Once you enter a real production environment, I generally assume one is required.

Especially in these scenarios: when user data is involved, when write operations are involved, when multi-tenancy is involved, when cost control is involved, or when integrating with internal enterprise systems.

I typically have the API Gateway handle the following tasks: validate user identity, inject tenant_id and user_id, check tool call scopes, add secondary confirmation for write operations, record request_id / tool_call_id / trace_id, enforce rate limits and cost limits, filter sensitive fields, route high-risk operations for manual review, return structured errors upon failure, and retain audit logs.

This logic closely mirrors that discussed in n8n Webhook Production Hardening.

A public-facing entry point should never just ask “is it reachable?” More importantly: who can invoke it? How many times? What exactly is being invoked? How are failures handled? And can the process be traced afterward?

8. A More Production-Ready Agent Architecture

If I were to build a production-grade AI Agent tool system today, I wouldn’t write it like this:

LLM → tools[] → 业务 API

This is too thin.

I’ll break it down like this:

用户请求
  ↓
Agent Runtime
  ↓
Function Calling / Tool Choice
  ↓
MCP Client
  ↓
MCP Server
  ↓
API Gateway
  ↓
业务服务 / 数据库 / 文件系统 / SaaS
  ↓
审计日志 / Trace / Cost Meter / Review Queue

Among these, the Agent Runtime handles sessions, task state, tool selection, and interruption recovery. Function Calling enables the model to specify which tool to invoke and what parameters to pass. The MCP Client connects to various MCP Servers. The MCP Server exposes tools, resources, prompts, and context capabilities. The API Gateway manages permissions, rate limiting, auditing, isolation, and degradation. Business services handle actual data reads and writes. The audit system preserves the chain of evidence.

This architecture may appear more complex than a demo, but its advantage is clear separation of concerns: the model never directly touches business systems; the MCP Server does not hold all permissions; the Gateway doesn’t understand model inference but controls access boundaries; every call is traceable; and failures can be pinpointed to a specific layer.

This has become increasingly clear to me while working on the XBSTACK tool page and the AI Finance evidence chain: building an AI application isn’t just about integrating a model. The real time sink is “the buffer layer between the model and real-world systems.”

9. My Decision Framework

Here is a decision framework I use myself.

If you are still building a demo: Function Calling is sufficient.

If you need to integrate many tools and want multiple clients to reuse them: Function Calling + MCP.

If you are serving real users, real data, and real write operations: Function Calling + MCP + API Gateway.

If you are asking yourself “Do I need MCP?”, you are still stuck at the protocol level. A better question is: Which layer is currently bottlenecking my system?

If the bottleneck is that the model doesn’t know how to express tool calls, it’s a Function Calling issue.

If the bottleneck is too many tools, scattered context, and difficulty reusing clients, it’s an MCP issue.

If the bottleneck is permissions, auditing, rate limiting, user isolation, cost, and production governance, it’s an API Gateway issue.

Once these three layers are clearly defined, an AI Agent stops being just a chatbot for demos and becomes an engineering component capable of integrating with real systems.

FAQ

If I already have Function Calling, do I still need MCP?

Not necessarily. When you have few tools, simple business logic, and only serve one application, Function Calling is enough. MCP is better suited for scenarios involving multiple tools, multiple resources, and client reuse.

Can MCP replace an API Gateway?

No. MCP standardizes the integration of tools and context, while an API Gateway handles authentication, rate limiting, auditing, isolation, and risk control in production environments. They operate at different layers.

Should the API Gateway sit in front of or behind the MCP Server?

Typically, it sits before the MCP Server accesses real business services. The MCP Server can expose tool capabilities, but permissions and auditing for real business systems should ideally still be managed by the Gateway or the business services themselves.

Do small projects really need this level of complexity?

No. For small projects, start with Function Calling to get the workflow running. However, it’s best to set up logging, separate read-only from write operations, and define failure handling from day one.

Which article on XBSTACK should be optimized first?

From an SEO and site structure perspective, I recommend starting with the main pillar page: MCP vs Function Calling: Differences, Use Cases, and When to Use MCP? . This article serves as its engineering architecture support piece, not a replacement.

Recommended Reading

• MCP vs Function Calling: Differences, Use Cases, and When to Use MCP
• MCP Server Production Governance: Permissions, Auditing, Tool Boundaries, and Security Policies
• MCP Security Best Practices: Prompt Injection, Tool Boundaries, and Audit Governance
• n8n Webhook Production Hardening: Implementing Signatures, Rate Limiting, and Idempotency for Public Endpoints
• AI Financial Report Analysis Assistant Development Roadmap: PDF Parsing, Task Queues, Evidence Chains, and Human Review

References

• Model Context Protocol Specification, version 2025-06-18
• OpenAI Function Calling Guide

Topic path / MCP

Continue from protocol details to production MCP governance

The MCP hub connects protocol fundamentals, transports, authentication, security, JSON-RPC debugging and production deployment without splitting the search intent across isolated guides.

Open MCP hub → Review the security checklist Subscribe for updates