Expose Nginx Load Balancer MetalLB with K3s

Overview

This guide demonstrates how to expose an Nginx web server using MetalLB load balancer in a K3s Kubernetes cluster. MetalLB is a load-balancer implementation for bare metal Kubernetes clusters that uses standard routing protocols, providing external IP addresses for services of type LoadBalancer.

Key Concepts

LoadBalancer Service Type: In Kubernetes, a Service of type LoadBalancer automatically provisions an external load balancer. On bare metal clusters without cloud provider integration, MetalLB fulfills this role by allocating external IPs from a configured pool and announcing them via Layer 2 (ARP/NDP) or BGP protocols.

MetalLB Operating Modes:

• Layer 2 Mode: Uses ARP (IPv4) or NDP (IPv6) to announce the external IP on the local network. Simple to configure, no router cooperation needed.
• BGP Mode: Integrates with network routers using Border Gateway Protocol for advanced routing scenarios and true load balancing across multiple nodes.

Prerequisites

Before deploying, ensure:

• K3s cluster is running (K3s includes Traefik by default, but MetalLB provides LoadBalancer IPs)
• MetalLB is installed and configured with an IPAddressPool
• kubectl is configured to access your cluster

Quick Deployment Methods

Method 1: Imperative Commands (Quick Testing)

1kubectl create deploy nginx --image nginx:latest
2kubectl expose deploy nginx --port 3000 --target-port 80 --type LoadBalancer

Command Breakdown:

• kubectl create deploy nginx: Creates a Deployment named "nginx" with default replica count (1)
• --image nginx:latest: Uses the latest Nginx image from Docker Hub
• kubectl expose deploy nginx: Creates a Service resource for the nginx Deployment
• --port 3000: External port that clients will connect to
• --target-port 80: Internal container port where Nginx listens
• --type LoadBalancer: Requests an external IP from MetalLB

Method 2: Declarative YAML (Production Recommended)

1apiVersion: apps/v1
2kind: Deployment
3metadata:
4  name: nginx
5spec:
6  replicas: 2
7  selector:
8    matchLabels:
9      app: nginx
10  template:
11    metadata:
12      labels:
13        app: nginx
14    spec:
15      containers:
16      - name: nginx
17        image: nginx:latest
18        ports:
19        - containerPort: 80
20---
21apiVersion: v1
22kind: Service
23metadata:
24  name: nginx
25spec:
26  type: LoadBalancer
27  selector:
28    app: nginx
29  ports:
30    - port: 3000
31      targetPort: 80

Manifest Explanation:

• replicas: 2: Runs two Nginx pod instances for high availability
• selector.matchLabels: Links Service to Pods with label app: nginx
• containerPort: 80: Exposes container port 80 (Nginx default)
• type: LoadBalancer: Triggers MetalLB to assign an external IP
• port: 3000: External service port
• targetPort: 80: Routes traffic to container port 80

Deployment and Verification

Deploy the Application

1# Apply the declarative manifest
2kubectl apply -f nginx-lb.yaml
3
4# Verify deployment status
5kubectl get deployments
6kubectl get pods -l app=nginx -o wide
7
8# Check service and external IP assignment
9kubectl get svc nginx

Expected Output:

The EXTERNAL-IP field shows the IP assigned by MetalLB from its configured pool.

Verify Connectivity

1# Test the external IP (replace with your assigned IP)
2curl http://192.168.1.240:3000
3
4# Check MetalLB speaker logs
5kubectl logs -n metallb-system -l component=speaker
6
7# Verify service endpoints
8kubectl get endpoints nginx

Advanced Configuration Options

Client IP Preservation

By default, traffic routed through a LoadBalancer may obscure the original client IP. To preserve source IPs, configure externalTrafficPolicy: Local:

1apiVersion: v1
2kind: Service
3metadata:
4  name: nginx
5spec:
6  type: LoadBalancer
7  externalTrafficPolicy: Local
8  selector:
9    app: nginx
10  ports:
11    - port: 3000
12      targetPort: 80

Benefits:

• Preserves client source IP addresses
• Traffic is routed directly to nodes hosting the pods
• Useful for logging, security policies, and IP-based access control

Trade-offs:

• Nodes without local endpoints are excluded from load balancing
• May result in uneven traffic distribution

Scaling the Deployment

Adjust the number of Nginx replicas based on load requirements:

1# Scale to 5 replicas
2kubectl scale deployment nginx --replicas=5
3
4# Verify scaled pods
5kubectl get pods -l app=nginx
6
7# Enable horizontal pod autoscaling (requires metrics-server)
8kubectl autoscale deployment nginx --min=2 --max=10 --cpu-percent=80

Autoscaling Behavior:

• Monitors CPU utilization across pods
• Scales up when average CPU exceeds 80%
• Scales down when demand decreases
• Maintains minimum 2 and maximum 10 replicas

Resource Limits and Requests

Define resource constraints for production environments:

1apiVersion: apps/v1
2kind: Deployment
3metadata:
4  name: nginx
5spec:
6  replicas: 2
7  selector:
8    matchLabels:
9      app: nginx
10  template:
11    metadata:
12      labels:
13        app: nginx
14    spec:
15      containers:
16        - name: nginx
17          image: nginx:latest
18          ports:
19            - containerPort: 80
20          resources:
21            requests:
22              memory: "64Mi"
23              cpu: "100m"
24            limits:
25              memory: "128Mi"
26              cpu: "200m"

Resource Management:

• requests: Guaranteed minimum resources for scheduling
• limits: Maximum resources the container can consume
• Prevents resource starvation and improves cluster stability

MetalLB Configuration Reference

Layer 2 Mode Configuration

1apiVersion: metallb.io/v1beta1
2kind: IPAddressPool
3metadata:
4  name: first-pool
5  namespace: metallb-system
6spec:
7  addresses:
8    - 192.168.1.240-192.168.1.250
9---
10apiVersion: metallb.io/v1beta1
11kind: L2Advertisement
12metadata:
13  name: l2-advert
14  namespace: metallb-system
15spec:
16  ipAddressPools:
17    - first-pool

Layer 2 Characteristics:

• Uses ARP/NDP for IP announcement
• All traffic for a service goes to one node (elected leader)
• Automatic failover if the leader node becomes unavailable
• No router configuration required

BGP Mode Configuration

1apiVersion: metallb.io/v1beta1
2kind: IPAddressPool
3metadata:
4  name: bgp-pool
5  namespace: metallb-system
6spec:
7  addresses:
8    - 192.168.1.240/29
9  autoAssign: true
10---
11apiVersion: metallb.io/v1beta2
12kind: BGPPeer
13metadata:
14  name: router-peer
15  namespace: metallb-system
16spec:
17  myASN: 64512
18  peerASN: 64512
19  peerAddress: 192.168.1.1
20---
21apiVersion: metallb.io/v1beta1
22kind: BGPAdvertisement
23metadata:
24  name: bgp-advert
25  namespace: metallb-system
26spec:
27  ipAddressPools:
28    - bgp-pool
29  communities:
30    - 65535:65282

BGP Mode Advantages:

• True load balancing across multiple nodes
• Better suited for production environments
• Integrates with existing network infrastructure
• Supports advanced routing policies and communities

Troubleshooting Guide

Service Not Getting External IP

1# Check MetalLB controller logs
2kubectl logs -n metallb-system -l app=metallb,component=controller
3
4# Verify IPAddressPool configuration
5kubectl get ipaddresspools -n metallb-system
6
7# Check L2Advertisement or BGPAdvertisement
8kubectl get l2advertisements -n metallb-system
9kubectl get bgpadvertisements -n metallb-system

Common Issues:

• IPAddressPool range exhausted or not defined
• Advertisement resource missing or misconfigured
• Network policy blocking MetalLB speaker communication

Pods Not Receiving Traffic

1# Check service endpoints
2kubectl describe svc nginx
3
4# Verify pod readiness
5kubectl get pods -l app=nginx
6
7# Test service internally
8kubectl run test-pod --rm -it --image=busybox -- wget -O- http://nginx:3000

Testing Network Connectivity

1# Check connectivity from another node
2curl http://<EXTERNAL-IP>:3000
3
4# Verify ARP table (Layer 2 mode)
5arp -a | grep <EXTERNAL-IP>
6
7# Check MetalLB speaker on specific node
8kubectl logs -n metallb-system <speaker-pod-name>

Production Best Practices

1. Use Declarative Manifests: Always use YAML files for reproducible deployments
2. Configure Resource Limits: Prevent resource exhaustion with proper limits
3. Enable Health Checks: Implement readiness and liveness probes
4. Version Pin Images: Use specific tags instead of :latest
5. Monitor External IPs: Track IP allocation and usage from MetalLB pools
6. Plan IP Ranges: Ensure IPAddressPool ranges don't conflict with DHCP or other services
7. Document Network Topology: Keep records of BGP ASNs, peer relationships, and IP assignments

Enhanced Deployment with Health Checks

1apiVersion: apps/v1
2kind: Deployment
3metadata:
4  name: nginx
5  labels:
6    app: nginx
7spec:
8  replicas: 3
9  selector:
10    matchLabels:
11      app: nginx
12  template:
13    metadata:
14      labels:
15        app: nginx
16    spec:
17      containers:
18        - name: nginx
19          image: nginx:1.25.3 # Pinned version
20          ports:
21            - containerPort: 80
22              protocol: TCP
23          resources:
24            requests:
25              memory: "64Mi"
26              cpu: "100m"
27            limits:
28              memory: "128Mi"
29              cpu: "200m"
30          livenessProbe:
31            httpGet:
32              path: /
33              port: 80
34            initialDelaySeconds: 30
35            periodSeconds: 10
36          readinessProbe:
37            httpGet:
38              path: /
39              port: 80
40            initialDelaySeconds: 5
41            periodSeconds: 5

Health Check Benefits:

• livenessProbe: Restarts container if it becomes unresponsive
• readinessProbe: Removes pod from service endpoints until ready
• Ensures traffic only reaches healthy pods
• Improves reliability during rolling updates

Additional Resources

• MetalLB Official Documentation
• Kubernetes Service Types
• K3s Documentation
• Nginx Docker Hub