Adopting the Kubernetes Gateway API: GatewayClass, HTTPRoute Traffic Splitting, and Migrating off Ingress

The Ingress resource froze in time. It has been v1 and effectively feature-complete since Kubernetes 1.19, which is why every controller bolted its real capabilities — canaries, header routing, rewrites, gRPC — onto a sprawl of nginx.ingress.kubernetes.io/* annotations that don’t port between vendors. The Gateway API is the official replacement: a typed, role-separated, expressive successor that graduated its core resources (GatewayClass, Gateway, HTTPRoute) to v1 (GA) in October 2023 and has been extended every release since. This guide takes you from the resource model through weighted traffic splitting and a dual-running cutover off Ingress — the way a platform team actually rolls it out.

Everything here targets Gateway API v1.x as it ships today. Commands are real and current; where a feature is still in v1alpha2/experimental I flag it rather than imply it is stable.

1. The resource model and why it is split three ways

Ingress conflates two audiences into one object: the cluster operator who owns the load balancer and TLS, and the app team who owns routing. Gateway API splits that into a layered model where each resource has one owner.

The chain is GatewayClass <- Gateway <- HTTPRoute -> Service. A GatewayClass names a controllerName (for example gateway.envoyproxy.io/gatewayclass-controller). A Gateway references a GatewayClass and declares listeners. An HTTPRoute attaches to a Gateway via parentRefs and forwards to backend Services.

Mental model: GatewayClass is the kind of load balancer available (like a StorageClass). Gateway is one provisioned instance of it. HTTPRoute is a tenant renting a hostname on that instance. Ownership, RBAC, and namespaces fall on those seams cleanly — which is the entire point.

2. Role separation and namespace boundaries

The split is not cosmetic; it changes who can grant what. Two controls enforce the boundary:

Gateway.spec.listeners[].allowedRoutes decides which namespaces may attach routes to a listener. ReferenceGrant decides whether an HTTPRoute in namespace A may forward to a Service in namespace B. Both default to closed: same-namespace only.

A platform team typically runs one shared Gateway and lets vetted app namespaces attach to it:

apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: shared-gateway
  namespace: gateway-infra
spec:
  gatewayClassName: envoy-gateway
  listeners:
    - name: https
      protocol: HTTPS
      port: 443
      hostname: "*.apps.kloudvin.io"
      tls:
        mode: Terminate
        certificateRefs:
          - kind: Secret
            name: apps-wildcard-tls
      allowedRoutes:
        namespaces:
          from: Selector
          selector:
            matchLabels:
              gateway-access: "true"

Now only namespaces carrying gateway-access: true can bind. An app team’s route in such a namespace references the shared Gateway across the namespace boundary:

apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: shop
  namespace: team-shop
spec:
  parentRefs:
    - name: shared-gateway
      namespace: gateway-infra
  hostnames:
    - shop.apps.kloudvin.io
  rules:
    - backendRefs:
        - name: shop-svc
          port: 8080

If shop-svc lived in a different namespace than the route, you would also need a ReferenceGrant in that target namespace authorizing the cross-namespace backendRef. Same-namespace forwarding, as above, needs none.

3. Install a conformant implementation

Gateway API is a set of CRDs plus a controller that implements them. The CRDs install once; the controller is your choice — Envoy Gateway, NGINX Gateway Fabric, Istio, Cilium, and the managed offerings (GKE Gateway, AWS Gateway API controller) are all conformant. I’ll use Envoy Gateway as it is a clean, dedicated implementation.

Install the standard-channel CRDs, then the controller:

# Standard channel: GA resources (Gateway, GatewayClass, HTTPRoute) + ReferenceGrant
kubectl apply -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.3.0/standard-install.yaml

# Envoy Gateway controller
helm install eg oci://docker.io/envoyproxy/gateway-helm \
  --version v1.4.0 -n envoy-gateway-system --create-namespace

kubectl wait --timeout=5m -n envoy-gateway-system \
  deployment/envoy-gateway --for=condition=Available

The CRDs ship in two channels. Standard carries GA resources. Experimental adds alpha fields and resources like TCPRoute, TLSRoute, and newer policy attachments. Pick one channel cluster-wide and never mix them — installing experimental over standard is fine, but reverting drops fields and can orphan objects.

Each implementation ships its own GatewayClass. Create one that binds to Envoy Gateway’s controller:

apiVersion: gateway.networking.k8s.io/v1
kind: GatewayClass
metadata:
  name: envoy-gateway
spec:
  controllerName: gateway.envoyproxy.io/gatewayclass-controller

Confirm the class is accepted before going further — an unaccepted class means nothing downstream will provision:

kubectl get gatewayclass envoy-gateway -o jsonpath='{.status.conditions[?(@.type=="Accepted")].status}{"\n"}'
# Expect: True

4. Listeners, TLS termination, and matching in HTTPRoute

A Gateway listener defines the L4/TLS entry point; the HTTPRoute does L7 matching. TLS is terminated at the Gateway via mode: Terminate referencing a Kubernetes Secret of type kubernetes.io/tls — the same secret shape Ingress used, so your existing cert-manager Certificate objects carry over untouched.

Matching in an HTTPRoute is far richer than Ingress paths. You match on path (Exact, PathPrefix, RegularExpression), headers, query params, and method, all within a single rule:

apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: api
  namespace: team-shop
spec:
  parentRefs:
    - name: shared-gateway
      namespace: gateway-infra
  hostnames:
    - api.apps.kloudvin.io
  rules:
    - matches:
        - path:
            type: PathPrefix
            value: /v2
          method: GET
      backendRefs:
        - name: api-v2
          port: 80
    - matches:
        - path:
            type: PathPrefix
            value: /
      backendRefs:
        - name: api-v1
          port: 80

Rule ordering is specification-defined, not first-match-wins guesswork. The spec mandates precedence: an exact path beats a prefix, a longer prefix beats a shorter one, and more matching criteria beat fewer. That determinism is a real upgrade over annotation-driven Ingress, where ordering was per-controller behavior you had to memorize.

5. Weighted backends: canary and blue-green

This is where the annotation era dies. Traffic splitting is a first-class field: list multiple backendRefs under one rule and assign each a weight. Weights are relative, not percentages — {90, 10} and {900, 100} are identical.

apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: shop-canary
  namespace: team-shop
spec:
  parentRefs:
    - name: shared-gateway
      namespace: gateway-infra
  hostnames:
    - shop.apps.kloudvin.io
  rules:
    - backendRefs:
        - name: shop-stable
          port: 8080
          weight: 90
        - name: shop-canary
          port: 8080
          weight: 10

Shift weight by patching the route — no controller-specific annotation, no rebuild:

kubectl -n team-shop patch httproute shop-canary --type=json -p='[
  {"op":"replace","path":"/spec/rules/0/backendRefs/0/weight","value":50},
  {"op":"replace","path":"/spec/rules/0/backendRefs/1/weight","value":50}
]'

A weight: 0 backend receives no traffic but stays a declared, ready target — exactly the blue-green primitive: keep both colors at weight, flip 100/0 to 0/100 in one apply, and roll back by reversing it. Because this is a plain Kubernetes field, progressive delivery controllers (Argo Rollouts, Flagger) drive these weights natively via the Gateway API plugin instead of templating vendor annotations.

For routing by client attributes rather than ratio, match on headers or query params and send each match to a different backend:

  rules:
    - matches:
        - headers:
            - name: x-canary
              value: "true"
      backendRefs:
        - name: shop-canary
          port: 8080
    - backendRefs:        # default rule, no matches
        - name: shop-stable
          port: 8080

Header value matching is Exact by default; set type: RegularExpression for patterns. This gives you opt-in canaries (internal users send x-canary: true) with zero blast radius on real traffic.

6. Filters: mirror, redirect, rewrite

Filters run on a rule’s matched requests. The portable ones in the standard channel:

• RequestHeaderModifier / ResponseHeaderModifier — set, add, or remove headers
• RequestMirror — fork a copy of traffic to a second backend, responses discarded
• RequestRedirect — issue 301/302 (scheme, host, port, path, status)
• URLRewrite — rewrite hostname or path before forwarding

Shadow production traffic to a new build to test it under real load without affecting responses:

  rules:
    - filters:
        - type: RequestMirror
          requestMirror:
            backendRef:
              name: shop-canary
              port: 8080
      backendRefs:
        - name: shop-stable
          port: 8080

RequestMirror is fire-and-forget: the mirror backend’s latency and errors never reach the client, which makes it the safest way to validate a release. Combine a path rewrite with the redirect-to-HTTPS pattern most teams need on day one:

  rules:
    # Strip /legacy prefix before forwarding
    - matches:
        - path: { type: PathPrefix, value: /legacy }
      filters:
        - type: URLRewrite
          urlRewrite:
            path:
              type: ReplacePrefixMatch
              replacePrefixMatch: /
      backendRefs:
        - name: shop-svc
          port: 8080

For the HTTP-to-HTTPS redirect, add a plain HTTP listener on port 80 and a tiny route whose only job is a RequestRedirect filter with scheme: https and statusCode: 301 and no backendRefs.

7. Side-by-side migration off Ingress

Do not flip DNS and pray. Run Gateway API beside Ingress, validate, then cut over. The kubernetes-sigs ingress2gateway tool converts existing Ingress (and several vendors’ annotations) into Gateway API YAML so you start from your real config, not a blank file.

# Install
go install github.com/kubernetes-sigs/ingress2gateway@v0.4.0

# Convert live Ingress in a namespace; --providers maps vendor annotations
ingress2gateway print --namespace team-shop --providers ingress-nginx > generated-gw.yaml

ingress2gateway translates what maps cleanly. Annotations with no Gateway API equivalent (auth snippets, rate limits, raw config blocks) are dropped or flagged, not silently faked. Treat the output as a reviewed first draft. Diff it, fill the gaps with the right policy attachment, and never apply it blind.

The dual-run sequence that avoids an outage:

1. Install CRDs + controller. The new Gateway provisions its own load balancer/IP, fully isolated from the Ingress LB.
2. Apply the converted Gateway + HTTPRoutes. Existing Ingress keeps serving production untouched.
3. Smoke-test the new path by sending traffic to the Gateway’s IP with the production Host header, bypassing DNS:

GW_IP=$(kubectl -n gateway-infra get gateway shared-gateway \
  -o jsonpath='{.status.addresses[0].value}')

curl -sS -k --resolve shop.apps.kloudvin.io:443:$GW_IP \
  https://shop.apps.kloudvin.io/healthz -o /dev/null -w "%{http_code}\n"

1. Cut over at DNS: repoint the hostname’s record from the Ingress LB to the Gateway address, or shift weight at a global load balancer for a gradual cutover.
2. Bake for a release cycle so rollback is a single DNS change, then delete the Ingress objects and controller.

The key safety property: the two data planes have separate IPs the whole time, so nothing about installing or testing the Gateway can perturb live Ingress traffic.

Verify

A Gateway API rollout has a precise, machine-readable health model — use the status conditions, not curl alone.

# 1. GatewayClass accepted by its controller
kubectl get gatewayclass envoy-gateway \
  -o jsonpath='{.status.conditions[?(@.type=="Accepted")].status}{"\n"}'

# 2. Gateway is Programmed (LB provisioned) and has an address
kubectl -n gateway-infra get gateway shared-gateway \
  -o jsonpath='Programmed={.status.conditions[?(@.type=="Programmed")].status} addr={.status.addresses[0].value}{"\n"}'

# 3. Per-listener attached route count — catches selector/namespace mistakes
kubectl -n gateway-infra get gateway shared-gateway \
  -o jsonpath='{range .status.listeners[*]}{.name}={.attachedRoutes}{"\n"}{end}'

# 4. THE critical check: is the route accepted AND resolved by its parent?
kubectl -n team-shop get httproute shop-canary \
  -o jsonpath='{range .status.parents[*]}Accepted={.conditions[?(@.type=="Accepted")].status} ResolvedRefs={.conditions[?(@.type=="ResolvedRefs")].status}{"\n"}{end}'

The single most common failure is an unattached route: the HTTPRoute exists but attachedRoutes on the Gateway stays 0 and the route reports Accepted=False. Read the condition reason:

• NotAllowedByListeners — the listener’s allowedRoutes rejects this route’s namespace. Fix the label/selector.
• NoMatchingParent / NoMatchingListenerHostname — parentRefs name, namespace, sectionName, or hostname don’t line up with any listener.
• ResolvedRefs=False with reason BackendNotFound or RefNotPermitted — the Service doesn’t exist, the port is wrong, or a cross-namespace backendRef lacks a ReferenceGrant.

kubectl describe httproute surfaces these reason/message pairs directly. Confirm the data plane end to end once status is green:

for i in $(seq 1 20); do
  curl -sS --resolve shop.apps.kloudvin.io:443:$GW_IP \
    https://shop.apps.kloudvin.io/version
done | sort | uniq -c   # ~90/10 split across stable/canary responses

Enterprise scenario

A retail platform team ran a single NGINX Ingress controller as a shared front door for ~120 app teams. The pain was governance: any team could ship an Ingress whose nginx.ingress.kubernetes.io/server-snippet annotation injected raw config into the shared NGINX, and one bad snippet had once reload-looped the controller and took down unrelated tenants. Ingress gave them no way to let teams self-serve routing while denying them control over the shared proxy.

They moved to Gateway API specifically for the ownership seam. Platform owned one Gateway per environment with allowedRoutes gated on a namespace label that only their admission policy could set. App teams got full HTTPRoute self-service — weighted canaries, header routing, rewrites — but HTTPRoute has no raw-config escape hatch, so a tenant could no longer reach into the shared data plane. Cross-namespace backends required an explicit ReferenceGrant that the target team had to author, turning an implicit trust into a reviewed one.

The constraint that nearly stalled them: roughly 30 of the converted Ingresses relied on external-auth and rate-limit annotations that ingress2gateway correctly dropped. Rather than block the migration on a single policy story, they kept those few routes on Ingress during the bake and reimplemented the auth as the implementation’s policy attachment (a SecurityPolicy referencing the route), migrating them last:

apiVersion: gateway.envoyproxy.io/v1alpha1
kind: SecurityPolicy
metadata:
  name: shop-extauth
  namespace: team-shop
spec:
  targetRefs:
    - group: gateway.networking.k8s.io
      kind: HTTPRoute
      name: shop-canary
  extAuth:
    http:
      backendRefs:
        - name: ext-authz
          port: 9000

Outcome: 90 of 120 teams migrated in the first two sprints on the portable core; the long tail rode the dual-run until their policy attachments landed. The win wasn’t routing features — it was that the new model made “self-service routing” and “no shared-proxy blast radius” the same design instead of opposing ones.

Note the API group on that SecurityPolicy: gateway.envoyproxy.io/v1alpha1. Policy attachment is implementation-specific and still alpha across vendors. The routing is portable Gateway API; the policy (auth, rate limiting, mTLS to backends) ties you to one implementation today. Plan that coupling deliberately.

Checklist

• Standard-channel CRDs installed at a pinned version; channel chosen cluster-wide
• GatewayClass reports Accepted=True for the chosen controller
• Shared Gateway gates attachment via allowedRoutes on a namespace label
• TLS terminated at the Gateway with an existing cert-manager Secret
• Canary expressed as weighted backendRefs; weight shifted by patch, not annotation
• Header/query matching used for opt-in canaries with zero blast radius
• RequestMirror validating the new build under real traffic before any weight shift
• Ingress converted with ingress2gateway and the diff reviewed, not applied blind
• New Gateway provisioned a separate LB/IP; dual-run smoke test via --resolve passed
• Every HTTPRoute checked for Accepted=True and ResolvedRefs=True per parent
• Cross-namespace backendRefs backed by an explicit ReferenceGrant
• Implementation-specific policy (auth, rate limit) re-homed as policy attachment, not left on dropped annotations
• DNS/global-LB cutover staged with a one-step rollback; Ingress kept for one bake cycle

Pitfalls

• Mixing CRD channels. Standard and experimental ship different field sets. Reverting from experimental drops fields and can orphan objects mid-flight. Pick one channel and pin it.
• Reading “route exists” as “route serves.” An HTTPRoute can be perfectly valid YAML and attach to nothing. attachedRoutes: 0 plus Accepted=False is the truth; the manifest applying cleanly is not.
• Forgetting ReferenceGrant for cross-namespace backends. Same-namespace forwarding is free; crossing a namespace silently fails with RefNotPermitted until the target namespace grants it.
• Treating weights as percentages. They are relative. {1, 1} is a 50/50 split, not 1%. Validate the realized ratio with the uniq -c loop, not the manifest.
• Assuming all features are portable. Routing, splitting, and the standard filters are portable. Auth, rate limiting, and backend mTLS are policy attachments that are implementation-specific and still alpha — that coupling is real, plan for it.
• Flipping DNS before status is green. The Gateway gets its own IP precisely so you can validate via --resolve first. Cut over only after conditions read healthy and the smoke test passes.

Next steps: wire the route status checks (Accepted, ResolvedRefs, attachedRoutes) into a CI gate so a broken HTTPRoute fails the PR instead of silently serving nothing, and hand canary weight control to Argo Rollouts via its Gateway API plugin so progressive delivery drives the weight fields instead of a human running kubectl patch.