Tackle MCP-Servers on Kubernetes with kmcp and AgentGateway

What is the “model context protocol” (MCP)

LLMs are gaining widespread adoption in our day-to-day lives. LLMs are good at generating and predicting text and even visuals, but lack the functionality to connect to external systems and alter them or use them to retrieve information. Thus, the model context protocol was created as a solution to establish a standard across all LLM providers for connecting their LLMs to external systems.

The model context protocol, or MCP, is a standard created by Anthropic back in late 2024. In the middle of 2025, it got an increasing adoption by OpenAI and Google. Its goal is to create a standardized way to let an LLM in an LLM-powered application access external systems. This enables your LLM to invoke functions and procedures (also referenced as “tool calls” in the MCP-Specification) or to retrieve information from different data sources, also called “resources” in the MCP-Specification. Although Anthropic, OpenAI, and other providers already provide a way to enable tool calling and resource retrieval, the MCP provides a standardized approach that works across different providers and frameworks in different applications. 

The MCP works in a client-server architecture. It usually contains the following actors:

• Host 
  This is our AI application, for example, a text editor, a code editor, or any other application that uses LLM technologies.
• LLM
  This is the LLM that the application uses. Can be any LLM such as OpenAI’s ChatGPT or Anthropic’s Claude or even an on-premises LLM.
• MCP-Client
  The MCP-Client is basically the “pipe” your LLM, MCP-Server and host use for communicating with each other. It pipes your LLM requests to the LLM and the LLM invokes the tool calls through the MCP-Client.
• MCP-Server
  The server contains all the tools and resource definitions and the actual implementation and code of the different tools. The logic for fetching resources is also included here.

In the following chart, you can find a standard MCP flow where the LLM uses a specific tool of an MCP-Server to fulfill the task. In this case, we assume that the host and the client are part of the same application. But it is also possible that the client and the host are fully decoupled.

As you can see here, the LLM doesn’t execute the tool directly, because LLMs are only capable of generating text and visuals. They rather generate text to the client saying that the LLM would like to call a tool with specific parameters and then, after users’ approval, the client calls the MCP-Server to execute the actual tool code. The communication between all actors in the MCP specification is done via JSON-RPC, whereas MCP-Servers can communicate via the standard input and standard output or via Streamable HTTP.

AgentGateway

In essence, MCP-Servers are software that can run in a container, which is great for workflows in Kubernetes or containerized workflows in general. But this might not apply to any MCP server. In Kubernetes workflows, a pod needs to expose an API or provide some kind of network reachability to expose it via a service, ingress, or gateway. For some MCP-Servers, this might not be directly possible because they can only communicate via standard input and standard output, which cannot be used to provide network reachability. In addition to that, because the MCP is a quite new and emerging technology, not every MCP-Server might implement essential enterprise features, such as authentication, authorization, and observability. Using a large number of different MCP-Servers might also pose the challenge of maintaining a large number of servers with different endpoints in your host application. 

These problems can be tackled with something called an “MCP-Gateway”, such as “AgentGateway”. This is an open-source gateway that is designed for AI-Agents and MCP-Servers. You can use it to extend your MCP-Servers with observability and security mechanisms to observe tool and resource interactions and calls or to guard your MCP-Servers with OAuth2. It can also provide a network transport layer for your stdio MCP-Servers, enabling them to communicate via Streamable HTTP and run properly on Kubernetes as well. Additionally, it supports the serving of LLMs and using the Agent-to-Agent (A2A) protocol, but these features won’t be covered in this blog.

To set up AgentGateway on Kubernetes, you need to use kgateway, an implementation of the Kubernetes gateway API. This can then be used to expose and serve your MCP-Severs on Kubernetes.

First, you need some kind of pod or deployment and a service for the MCP-Server, so it is reachable via the network. Then you need to create a so-called “Backend” of type MCP with kgateway. The examples use kgateway with version 2.2.0 and AgentGateway with version 0.10.5:

apiVersion: gateway.kgateway.dev/v1alpha1
kind: Backend
metadata:
  name: mcp-backend
spec:
  type: MCP
  mcp:
    name: mcp-server
    targets:
    - static:
        name: the-mcp-server
        host: mcp-service.default.svc.cluster.local
        port: 12001
        protocol: StreamableHTTP

This example uses a so-called “static” MCP backend where you need to provide the exact service hostname. Kgateway also supports “dynamic” MCP backends where the backend resource is attached to the service via label selectors.

Next, you need to create an HTTP route, a native resource of the Kubernetes Gateway API, which refers to our backend we just created:

apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: mcp
spec:
  parentRefs:
  - name: agentgateway
  rules:
    - backendRefs:
      - name: mcp-backend
        group: gateway.kgateway.dev
        kind: Backend

And at last, you need to create the Gateway resource. In a cloud environment, a cloud load balancer will be created, which then exposes your gateway.

apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: agentgateway
spec:
  gatewayClassName: agentgateway
  listeners:
  - protocol: HTTP
    port: 8080
    name: http
    allowedRoutes:
      namespaces:
        from: All

Now your MCP-Server should be reachable on port 8080. You can use the /mcp endpoint to connect your agents via Streamable HTTP.

One current drawback is that the AgentGateway setup with kgateway does not support all features, such as wrapping stdio servers or properly setting up MCP-Authentication. This will most likely change in future releases of kgateway. It also does not support being deployed with the ingress API. To use these important features in Kubernetes, we will look at a tool called kmcp. 

kMCP

apiVersion: kagent.dev/v1alpha1
kind: MCPServer
metadata:
  name: github-stdio-mcp
spec:
  deployment:
    image: "ghcr.io/github/github-mcp-server:latest"
    port: 12001
    cmd: "/server/github-mcp-server"
    args: ["stdio"]
    secretRefs:
        - name: github-token
  transportType: "stdio"

binds:
- port: 8080
  listeners:
  - routes:
      - backends:
          - mcp:
              targets:
              - name: mcp
                mcp:
                  host: http://localhost:8081/mcp/
        matches:
          - path:
              exact: /github/mcp
          - path:
              exact: /.well-known/oauth-protected-resource/github/mcp
          - path:
              exact: /.well-known/oauth-authorization-server/github/mcp
          - path:
              exact: /.well-known/oauth-authorization-server/github/mcp/client-registration
        name: mcp-github
        policies:
          cors:
            allowHeaders:
            - "*"
            allowOrigins:
            - "*"
          mcpAuthentication:
            issuer: http://keycloak.192.168.49.2.nip.io/realms/mcp
            jwksUrl: http://keycloak.192.168.49.2.nip.io/realms/mcp/protocol/openid-connect/certs
            audience: mcp_proxy
            provider:
              keycloak: {}
            resourceMetadata:
              resource: http://localhost:8080/github/mcp
              scopesSupported:
              - profile
              - offline_access
              - openid
              - email
              - roles
              bearerMethodsSupported:
              - header
              - body
              resourceDocumentation: http://localhost:8080/github/mcp/docs
          mcpAuthorization:
            rules:
            # Allow anyone to call “get_me”
            - 'mcp.tool.name == "get_me"'
            # Only the admin user can call “delete_repository”
            - 'jwt.sub = "admin" && mcp.tool.name == "delete_repository"'