AI governance vs MCP gateway vs AI gateway: three layers teams should not confuse

As enterprise AI architecture matures, the word “gateway” is being used for several different control points. At the same time, governance platforms are moving closer to runtime.

The concepts overlap, but they should not be collapsed.

An AI or LLM gateway typically sits between applications and model providers.

Common responsibilities include model routing, provider abstraction, API keys, quotas, rate limits, usage and cost telemetry, prompt or response policies, fallback, caching and model observability.

Its primary boundary is the model call.

An MCP gateway sits between an agent client and tools exposed through the Model Context Protocol.

It can aggregate MCP servers, filter available tools, handle authentication and provide a stable endpoint to different agent clients.

Its primary boundary is the tool call.

Arcade, Pipedream and other platforms are building substantial infrastructure around this layer because agents need more than raw MCP protocol support in production.

AI governance answers broader organizational questions:

• What AI systems exist?
• Who owns them?
• What risk applies?
• Which policies govern them?
• Which data and tools are connected?
• What evaluations have passed?
• Which exceptions exist?
• What happened in production?
• Can the organization prove why a decision was permitted?

Its primary boundary is accountability across the lifecycle.

Combining responsibilities can make setup simple, but it can also blur trust boundaries.

For example, a governance service does not need to hold every provider credential. A connection gateway does not need to become the system of record for every policy. A model gateway should not automatically become the authorization authority for business actions.

Separating these responsibilities can make the system easier to reason about and reduce the impact of a compromise.

Outborn’s architecture uses distinct layers:

Outborn Agent Platform — agent experiences, reasoning, analysis and orchestration.

Outborn Agent Connect — provider credentials, MCP connections, tools and execution.

Outborn Governance — inventory, policy, risk, observations, approvals and evidence.

Outborn Account — human identity, membership and application access.

The layers are intended to cooperate without duplicating responsibility.

An agent proposes a production deployment.

The AI runtime can prepare the requested change. Agent Connect can identify the provider capability and execution target. Governance can evaluate policy and determine that production deployment requires approval. A human approves the concrete proposal. Agent Connect executes within the approved scope. Evidence records the decision and outcome.

That is a different architecture from putting a broad deployment token directly into the agent runtime.

Draw every trust boundary and answer:

1. Where does the model key live?
2. Where do SaaS and infrastructure credentials live?
3. Who decides which tools an agent can see?
4. Who decides whether an action is allowed?
5. Where does approval happen?
6. Which service executes the action?
7. Where is audit evidence stored?
8. Which layer owns agent inventory?
9. Can each layer be changed independently?
10. Does failure in one layer expose unnecessary secrets or permissions?

The goal is not architectural purity. It is making authority and accountability explicit.

• Workato documentation
• Arcade MCP Gateway
• IBM watsonx.governance

A gateway is an enforcement point, not automatically the authority that defines enterprise policy. Trace a request from its initiating human through delegated identity, the model service, proposed tool parameters and the business application. At each boundary, document who authenticates the caller, who authorizes the specific action and who retains evidence. A protocol-compliant connector does not establish permission to modify every business record.

Boundary	Typical question	Evidence to retain
Model inference	May this request use this provider and model?	Model identifier, rate/cost decision, sanitized trace
Tool discovery	Is this tool available to this actor?	Authorized tool list and connection identity
Business execution	May this exact actor change this exact target?	Object-level authorization, approved parameters, result
Governance	Who owns the system and which policy applies?	Risk record, control mapping, exception and audit trail

Figure: an illustrative enterprise control architecture; not a statement that every vendor ships all of these components.

Consider an employee asking an agent to update a single sales opportunity. The connector can have valid OAuth credentials while the employee lacks access to that particular account. If the system only checks whether the updateOpportunity tool is allowed, it may confuse a valid connection with valid business authority. A safer execution service evaluates the exact record identifier, field changes, actor, delegated authority and current policy before executing. For high-impact changes, it should bind approval to the proposed arguments and revalidate authorization if the scope or permissions change.

This is also why a gateway outage needs an explicit failure policy: a low-risk model response may continue through an alternative provider, but a privileged write should not proceed when its authorization cannot be established.

For a proof of concept, test the same four cases in every architecture: permitted read, denied cross-tenant write, approval-required mutation and revocation while a retry is pending. Score systems on correct decisions, evidence completeness, credential isolation, recovery and user experience. A feature checklist without executable scenarios is weaker evidence than a recorded trial.

The distinction in this article is analytical rather than a universal vendor taxonomy; gateways may package overlapping capabilities. Outborn product responsibilities described above are architectural intent, not an independently verified compliance certification.

• MCP authorization security considerations
• NIST AI Risk Management Framework
• OpenTelemetry GenAI agent spans — development conventions

Research method: architecture analysis using published specifications and guidance; scenarios are illustrative, not vendor benchmark measurements.