Migrating from Ingress-Nginx to Gateway API with KubeLB

Introduction

The Kubernetes Ingress API served the community well for years, but its limitations are well documented. There is no support for traffic splitting, header-based routing, or TCP/UDP routing. Everything gets crammed into annotations that differ between controllers. If you have ever tried to port an Ingress resource from one controller to another, you know the pain of translating controller-specific annotations that were never designed to be portable.

Gateway API is the official successor to Ingress. It is a more expressive, role-oriented API that separates infrastructure concerns (Gateway) from application routing (HTTPRoute). Since reaching GA with v1.0, it has become the standard that the entire Kubernetes networking ecosystem is converging on.

On bare metal and self-managed clusters, this migration is harder than on cloud. There is no managed load balancer to handle the transition. You cannot just flip a switch in your cloud provider’s console. KubeLB fills this gap as a Kubernetes-native load balancer designed specifically for bare metal and multi-cluster environments. It provides a Gateway API implementation that works where most others do not.

This tutorial walks you through a complete migration from Ingress-Nginx to Gateway API using KubeLB, including TLS certificate migration and a safe cutover strategy. By the end, you will have a production-ready Gateway API setup running alongside your existing Ingress-Nginx controller, with a clear path to decommission it.

Why Migrate?

Before starting the migration, here is why it matters for your infrastructure.

Annotations are a dead end. Ingress-Nginx relies heavily on annotations for advanced routing. Need URL rewrites? That is nginx.ingress.kubernetes.io/rewrite-target. Rate limiting? nginx.ingress.kubernetes.io/limit-rps. These annotations are controller-specific and non-portable. When you switch controllers or need to support multiple, you start from scratch. This is not a theoretical concern — it is a daily operational burden for teams managing multiple clusters.

Gateway API is the official Kubernetes standard. It graduated to GA with v1.0 and is where the ecosystem is moving. Major projects, cloud providers, and service meshes are building Gateway API support. Staying on Ingress means falling behind the ecosystem and missing out on new features that will only ship as Gateway API extensions.

Role-oriented design solves real organizational problems. Gateway API separates concerns cleanly. Your infrastructure team manages Gateways — the entry points, TLS termination, and listener configuration. Application teams manage HTTPRoutes — the routing rules for their services. This maps naturally to how most organizations already work, but Ingress forced everything into a single resource that both teams had to touch.

KubeLB makes this possible on bare metal. Most Gateway API implementations are tied to specific cloud providers — AWS Gateway Controller, GKE’s implementation, Azure Application Gateway. KubeLB provides a Gateway API implementation specifically designed for bare metal and multi-cluster setups. If you are running on-premises or in a colocation facility, KubeLB is how you get Gateway API without a cloud provider dependency.

Step 1: Audit Your Current Ingress Resources

Before migrating anything, you need a clear picture of what you have. Surprises during migration come from Ingress resources you forgot about or annotations you did not account for.

Start by listing all Ingress resources across every namespace:

# List all Ingress resources across namespaces
kubectl get ingress -A

Export everything for reference. You will need this backup both as documentation and as a rollback artifact:

# Export each Ingress for reference
kubectl get ingress -A -o yaml > ingress-backup.yaml

For each Ingress resource, document the following:

• Hostname and paths
• TLS configuration (certificate secrets)
• Backend services and ports
• Any annotations (rate limiting, rewrites, auth, etc.)

Create a migration checklist that tracks your progress:

Tip: Pay special attention to Ingress-Nginx-specific annotations. Some, like nginx.ingress.kubernetes.io/rewrite-target, have direct Gateway API equivalents (the URLRewrite filter). Others, like nginx.ingress.kubernetes.io/auth-url for external authentication, may require custom solutions or implementation-specific extensions. Identify these early so they do not block your migration later.

Take note of how many unique TLS certificates you have and how they are managed. If cert-manager provisions them, the migration is straightforward. If you manually manage certificate secrets, you will need to plan for that separately.

Step 2: Install Gateway API CRDs

Gateway API is not installed by default in most Kubernetes clusters. You need to install the Custom Resource Definitions (CRDs) before you can create any Gateway API resources.

Install the standard channel CRDs:

kubectl apply -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.1.0/standard-install.yaml

Verify that the CRDs are installed:

kubectl get crd | grep gateway

You should see at least three CRDs:

• gateways.gateway.networking.k8s.io
• httproutes.gateway.networking.k8s.io
• gatewayclasses.gateway.networking.k8s.io

If you also need TCP or UDP routing (for database connections or game servers, for example), install the experimental channel instead, which includes TCPRoute and UDPRoute CRDs. For most web application migrations from Ingress-Nginx, the standard channel is sufficient.

Step 3: Install KubeLB

With the Gateway API CRDs in place, install KubeLB as your Gateway API implementation. KubeLB acts as the controller that watches for Gateway and HTTPRoute resources and configures the underlying load balancer infrastructure.

Add the KubeLB Helm repository and install the manager:

helm repo add kubelb https://charts.kubelb.io
helm repo update

helm install kubelb-manager kubelb/kubelb-manager \
  --namespace kubelb-system \
  --create-namespace \
  --set gatewayAPI.enabled=true

Verify that KubeLB is running:

kubectl get pods -n kubelb-system

All pods should be in Running state. KubeLB automatically registers itself as a GatewayClass:

kubectl get gatewayclass

You should see a kubelb GatewayClass with an Accepted status condition. This tells Kubernetes that KubeLB is ready to serve as a Gateway API implementation.

Warning: If you set useLoadBalancerClass: true in KubeLB’s configuration, make sure it targets only the LoadBalancer services you intend. A misconfiguration here causes KubeLB to manage ALL LoadBalancer services in the cluster, which can conflict with existing services and cause IP exhaustion. Explicitly scope KubeLB to its own LoadBalancerClass to avoid this problem. This is the single most common deployment mistake with KubeLB.

Step 4: Create Your First Gateway

A Gateway is the entry point for traffic. Think of it as the declarative equivalent of the Ingress controller itself — it defines where traffic enters the cluster and how it is terminated.

Create a Gateway resource:

apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: main-gateway
  namespace: default
  annotations:
    cert-manager.io/cluster-issuer: letsencrypt-prod
spec:
  gatewayClassName: kubelb
  listeners:
    - name: http
      protocol: HTTP
      port: 80
      allowedRoutes:
        namespaces:
          from: All
    - name: https
      protocol: HTTPS
      port: 443
      tls:
        mode: Terminate
        certificateRefs:
          - name: wildcard-tls
            kind: Secret
      allowedRoutes:
        namespaces:
          from: All

There are several important differences from Ingress worth calling out here:

• Listeners are separate from routing rules. The Gateway defines ports and protocols. The routing rules live in HTTPRoute resources. This separation is the foundation of Gateway API’s role-oriented design.
• TLS termination is configured on the Gateway, not on individual routes. This makes sense operationally — the infrastructure team controls TLS, and application teams do not need to think about it.
• allowedRoutes.namespaces.from: All lets HTTPRoutes from any namespace attach to this Gateway. In production, you may want to restrict this to specific namespaces using label selectors for tighter access control.

Apply the Gateway and wait for it to receive an IP address from KubeLB:

kubectl apply -f gateway.yaml
kubectl get gateway main-gateway

The ADDRESS column in the output should show an IP address once KubeLB has provisioned the load balancer. This is the IP you will eventually point your DNS records to.

Step 5: Convert Ingress Resources to HTTPRoutes

This is the core of the migration. You need to convert each Ingress resource into one or more HTTPRoute resources. Here is a systematic conversion pattern.

Before (Ingress):

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: app-ingress
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
    nginx.ingress.kubernetes.io/ssl-redirect: "true"
spec:
  tls:
    - hosts:
        - app.example.com
      secretName: app-tls
  rules:
    - host: app.example.com
      http:
        paths:
          - path: /api
            pathType: Prefix
            backend:
              service:
                name: api-service
                port:
                  number: 8080
          - path: /
            pathType: Prefix
            backend:
              service:
                name: frontend-service
                port:
                  number: 3000

After (HTTPRoute):

apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: app-route
spec:
  parentRefs:
    - name: main-gateway
  hostnames:
    - app.example.com
  rules:
    - matches:
        - path:
            type: PathPrefix
            value: /api
      filters:
        - type: URLRewrite
          urlRewrite:
            path:
              type: ReplacePrefixMatch
              replacePrefixMatch: /
      backendRefs:
        - name: api-service
          port: 8080
    - matches:
        - path:
            type: PathPrefix
            value: /
      backendRefs:
        - name: frontend-service
          port: 3000

Notice how the HTTPRoute is both more explicit and more structured. There is no ambiguity about what the rewrite does — it is a typed filter, not a regex-laden annotation. The parentRefs field explicitly declares which Gateway this route attaches to, making the relationship between infrastructure and routing clear.

Here is a reference table for converting common patterns:

Apply each converted HTTPRoute:

kubectl apply -f app-route.yaml

Verify that the route attached successfully:

kubectl get httproute app-route -o yaml

Check status.parents — it should show your Gateway with an Accepted condition set to True.

Step 6: Handle TLS Certificates

TLS certificate migration is often the trickiest part of moving from Ingress to Gateway API, especially if you are using Let’s Encrypt with DNS01 challenges. There are two approaches depending on your setup.

Option A: Reuse Existing Secrets

If cert-manager already manages your certificates, the same Kubernetes secrets work with Gateway API. You do not need to re-issue certificates. Reference the existing secret in the Gateway listener:

listeners:
  - name: https
    protocol: HTTPS
    port: 443
    tls:
      mode: Terminate
      certificateRefs:
        - name: existing-tls-secret

This is the simplest path. The certificates continue to be renewed by cert-manager through the existing Certificate resources, and the Gateway just reads the same secret.

Option B: Use cert-manager Gateway API Integration

cert-manager natively supports Gateway API. If you want cert-manager to manage certificates based on your Gateway configuration, annotate the Gateway:

metadata:
  annotations:
    cert-manager.io/cluster-issuer: letsencrypt-prod

With this annotation, cert-manager watches the Gateway’s listeners and automatically provisions certificates for the hostnames defined in your HTTPRoutes. This is the cleaner long-term approach because it ties certificate lifecycle to the Gateway rather than to legacy Ingress resources.

DNS01 Challenge Considerations

When switching from Ingress to Gateway API, your DNS records change because the new Gateway gets a different IP address from KubeLB. If you use DNS01 challenges for certificate validation, you need external-dns properly synchronized to update DNS records for the new IP.

Warning: DNS01 challenges frequently fail during migration if external-dns is not configured to update records for the new Gateway IP. Before cutting over, verify that external-dns has created the correct A/AAAA records for the Gateway’s IP address. Use dig +short app.example.com to confirm the IP matches your Gateway’s address. A mismatch here means certificates will not renew, and you will face downtime when current certificates expire.

If you use HTTP01 challenges instead, the challenge solver needs to be reachable through the Gateway. Verify that cert-manager’s HTTP01 solver pods are included in your HTTPRoute configuration or can receive traffic through the Gateway.

Step 7: Run Both in Parallel

Do not do a “big bang” cutover. The safest migration strategy runs Ingress-Nginx and Gateway API simultaneously, with DNS controlling which path receives production traffic.

Here is the parallel running strategy:

1. Keep your existing Ingress resources active. They continue serving production traffic through Ingress-Nginx. Nothing changes for your users yet.

2. Deploy HTTPRoutes pointing to the same backend services. Both the Ingress and HTTPRoute resources route to identical backends. The services do not need to change.

3. Use DNS to control which path receives traffic. Point a test subdomain to the Gateway IP for validation:

# Point a test subdomain to the Gateway IP for validation
# test-app.example.com → Gateway IP (KubeLB)
# app.example.com → Ingress-Nginx IP (unchanged)

4. Test thoroughly through the Gateway endpoint. Verify all routes, TLS termination, rewrites, and any other behavior you depend on. Run your integration tests against the test subdomain. Check response headers to confirm traffic is flowing through KubeLB and not Ingress-Nginx.

5. When confident, switch the production DNS to the Gateway IP. This is the actual cutover moment, and it is controlled entirely through DNS — not Kubernetes resource changes.

This approach gives you a clean rollback path at every stage. If something goes wrong after switching DNS, you point it back to the Ingress-Nginx IP. No Kubernetes changes required.

Step 8: Cutover and Decommission Ingress-Nginx

Once all traffic flows through the Gateway and you have validated that everything works correctly, it is time to clean up.

Switch your production DNS records to point to the Gateway IP:

# Switch DNS: app.example.com → Gateway IP
# Wait for DNS propagation (TTL-dependent, typically 5 minutes to 24 hours)

Verify that traffic is flowing through the Gateway:

# Check KubeLB manager logs for incoming traffic
kubectl logs -n kubelb-system -l app=kubelb-manager --tail=100

After confirming that production traffic flows correctly through Gateway API, remove the old Ingress resources:

# Remove old Ingress resources
kubectl delete ingress app-ingress

Wait for a validation period before removing the Ingress-Nginx controller entirely:

# After validation period (1-2 weeks), remove Ingress-Nginx controller
helm uninstall ingress-nginx -n ingress-nginx
kubectl delete namespace ingress-nginx

Tip: Keep Ingress-Nginx installed for 1-2 weeks after cutover as a rollback option. Only decommission it after you are confident that all routes work correctly through Gateway API. The cost of running an idle Ingress-Nginx controller is negligible compared to the cost of not having a rollback path.

Troubleshooting

KubeLB Managing Unintended LoadBalancer Services

If useLoadBalancerClass is misconfigured, KubeLB attempts to manage all LoadBalancer services in the cluster, causing IP conflicts and potential service disruption.

Fix: Explicitly set the LoadBalancerClass on services you want KubeLB to manage:

spec:
  loadBalancerClass: kubelb.io/kubelb

This ensures KubeLB only touches services that explicitly opt in. Review all existing LoadBalancer services in the cluster before enabling this feature.

Envoy xDS Keep-Alive Disconnects

Symptoms: Routes stop updating after configuration changes. Envoy logs show Connection is closed by peer during connecting.

Fix: Check KubeLB manager pod logs for xDS connection errors. Restart the manager pod if the xDS connection is stale:

kubectl rollout restart deployment kubelb-manager -n kubelb-system

For persistent issues, increase the xDS keep-alive interval in KubeLB’s Helm values configuration. This is more common in environments with aggressive network timeouts or firewalls between nodes.

DNS01 Certificate Validation Failures

Symptoms: Certificates stuck in Pending state. cert-manager logs show DNS01 challenge failures.

Fix: Verify external-dns is creating TXT records for the ACME challenge:

dig _acme-challenge.app.example.com TXT

Check external-dns logs for errors. Common causes include incorrect cloud provider credentials, DNS zone delegation issues, or the external-dns controller not watching Gateway resources (it needs to be configured with the --source=gateway-httproute flag).

HTTPRoute Not Attaching to Gateway

Symptoms: HTTPRoute shows status.parents as empty or with an error condition.

Fix: Check that the HTTPRoute’s parentRefs matches the Gateway name and namespace exactly. Verify the Gateway’s allowedRoutes permits routes from the HTTPRoute’s namespace. A common mistake is creating the HTTPRoute in a namespace that the Gateway does not allow:

kubectl get httproute app-route -o jsonpath='{.status.parents}' | jq .

If the conditions show NotAllowed, update the Gateway’s allowedRoutes to include the HTTPRoute’s namespace, or move the HTTPRoute to a permitted namespace.

Common Annotation Migration Reference

This table covers the most frequently used Ingress-Nginx annotations and their Gateway API equivalents:

Warning: Not all Ingress-Nginx annotations have direct Gateway API equivalents. Features like external auth (auth-url) and advanced rate limiting may require implementation-specific extensions or custom filters. Audit these during Step 1 and plan alternative implementations before starting the migration. Do not discover missing functionality during the cutover.

Next Steps

• What is KubeLB? (coming soon) — understand KubeLB’s architecture and how it manages load balancing across clusters
• KubeLB with KubeOne: Complete Bare Metal Networking Stack (coming soon) — combine KubeLB with KubeOne for a full bare metal networking solution
• Bare Metal Kubernetes with KubeOne and Terraform — set up the cluster that KubeLB load-balances

Summary

You migrated from Ingress-Nginx to Gateway API using KubeLB as the implementation. The key steps were: audit existing Ingress resources, install Gateway API CRDs and KubeLB, create a Gateway with listeners, convert Ingress resources to HTTPRoutes, migrate TLS certificates, run both systems in parallel, and cut over with DNS.

The biggest risks in this migration are DNS01 challenge failures during certificate migration and KubeLB’s LoadBalancerClass scope misconfiguration. Both are avoidable with proper preparation during the audit phase.

The parallel running strategy is non-negotiable for production environments. Run both systems simultaneously for at least a week before decommissioning Ingress-Nginx. The cost of running an idle controller is nothing compared to the cost of a failed migration with no rollback path.

Gateway API gives you a more expressive, portable, and maintainable networking layer. KubeLB makes it work on bare metal where other implementations cannot. Together, they provide a production-grade networking stack that scales with your infrastructure needs.