Selenium Grid on Kubernetes: Scalable Browser Test Infrastructure

Selenium Grid on Kubernetes: Scalable Browser Test Infrastructure

Kubernetes is the natural home for Selenium Grid at scale. The Hub is a stateless service that belongs behind a ClusterIP or LoadBalancer. The Nodes are stateless workers that should scale up when there's a test backlog and scale down when the queue is empty. Kubernetes handles exactly this workload model through Deployments, Services, and the Horizontal Pod Autoscaler.

This guide covers deploying Grid 4 to Kubernetes from scratch — writing the raw manifests so you understand what each piece does, then layering in the official Helm chart for production use, HPA configuration for automatic scaling, RBAC for security, and ingress for external access.

Prerequisites

You need a running Kubernetes cluster (local kind/minikube or cloud-managed), kubectl configured, and Helm 3 installed. Verify:

kubectl version --client
# Client Version: v1.28.4

helm version
# version.BuildInfo{Version:"v3.13.2"...}

kubectl cluster-info
# Kubernetes control plane is running at https://...

For local development, kind is the fastest path:

kind create cluster --name selenium-grid
kubectl config use-context kind-selenium-grid

Understanding the Deployment Model

Grid 4 on Kubernetes maps cleanly to Kubernetes primitives:

  • HubDeployment (1 replica, stateless) + Service (ClusterIP for internal, LoadBalancer/Ingress for external)
  • NodesDeployment (replicas scale with test demand) + Service per Node pod (required for Hub→Node session forwarding)
  • Event Bus portsService exposing ports 4442 and 4443

The tricky part is Node service discovery: when a Node registers with the Hub, it must advertise a hostname that the Hub can route back to. In Kubernetes, each Node pod needs its own Service or you need to use the pod's DNS name directly. The official approach uses SE_NODE_HOST environment variable pointing to the pod's own IP, which Kubernetes injects via the Downward API.

Raw Kubernetes Manifests

Start with the Hub:

# hub-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: selenium-hub
  namespace: selenium
  labels:
    app: selenium-hub
spec:
  replicas: 1
  selector:
    matchLabels:
      app: selenium-hub
  template:
    metadata:
      labels:
        app: selenium-hub
    spec:
      containers:
        - name: selenium-hub
          image: selenium/hub:4.18.1-20240224
          ports:
            - containerPort: 4444
            - containerPort: 4442
            - containerPort: 4443
          env:
            - name: SE_SESSION_REQUEST_TIMEOUT
              value: "300"
            - name: SE_SESSION_RETRY_INTERVAL
              value: "5"
          resources:
            requests:
              memory: "512Mi"
              cpu: "250m"
            limits:
              memory: "1Gi"
              cpu: "1000m"
          livenessProbe:
            httpGet:
              path: /status
              port: 4444
            initialDelaySeconds: 30
            periodSeconds: 15
            failureThreshold: 3
          readinessProbe:
            httpGet:
              path: /status
              port: 4444
            initialDelaySeconds: 15
            periodSeconds: 10
# hub-service.yaml
apiVersion: v1
kind: Service
metadata:
  name: selenium-hub
  namespace: selenium
spec:
  selector:
    app: selenium-hub
  ports:
    - name: web
      port: 4444
      targetPort: 4444
    - name: publish
      port: 4442
      targetPort: 4442
    - name: subscribe
      port: 4443
      targetPort: 4443
  type: ClusterIP

Now the Chrome Node Deployment, using the Downward API to inject the pod's IP as SE_NODE_HOST:

# chrome-node-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: selenium-node-chrome
  namespace: selenium
  labels:
    app: selenium-node-chrome
spec:
  replicas: 2
  selector:
    matchLabels:
      app: selenium-node-chrome
  template:
    metadata:
      labels:
        app: selenium-node-chrome
    spec:
      volumes:
        - name: dshm
          emptyDir:
            medium: Memory
            sizeLimit: 2Gi
      containers:
        - name: selenium-node-chrome
          image: selenium/node-chrome:4.18.1-20240224
          ports:
            - containerPort: 5555
          volumeMounts:
            - mountPath: /dev/shm
              name: dshm
          env:
            - name: SE_EVENT_BUS_HOST
              value: selenium-hub
            - name: SE_EVENT_BUS_PUBLISH_PORT
              value: "4442"
            - name: SE_EVENT_BUS_SUBSCRIBE_PORT
              value: "4443"
            - name: SE_NODE_HOST
              valueFrom:
                fieldRef:
                  fieldPath: status.podIP   # Downward API — pod's own IP
            - name: SE_NODE_MAX_SESSIONS
              value: "4"
            - name: SE_NODE_SESSION_TIMEOUT
              value: "300"
            - name: SE_NODE_OVERRIDE_MAX_SESSIONS
              value: "true"
            - name: SE_SCREEN_WIDTH
              value: "1920"
            - name: SE_SCREEN_HEIGHT
              value: "1080"
          resources:
            requests:
              memory: "1Gi"
              cpu: "500m"
            limits:
              memory: "3Gi"
              cpu: "2000m"
          livenessProbe:
            httpGet:
              path: /status
              port: 5555
            initialDelaySeconds: 30
            periodSeconds: 15

Apply the namespace and manifests:

kubectl create namespace selenium
kubectl apply -f hub-deployment.yaml
kubectl apply -f hub-service.yaml
kubectl apply -f chrome-node-deployment.yaml

kubectl -n selenium get pods
# NAME                                    READY   STATUS    RESTARTS   AGE
# selenium-hub-7d9b8f6c4-xk2vp           1/1     Running   0          45s
# selenium-node-chrome-5b8d9f7c3-4nmq6   1/1     Running   0          30s
# selenium-node-chrome-5b8d9f7c3-p8j7r   1/1     Running   0          30s

Using the Official Helm Chart

The official docker-selenium Helm chart encapsulates all the manifests above with sensible defaults and extensive configuration options:

helm repo add docker-selenium https://www.selenium.dev/docker-selenium
helm repo update

# Inspect default values before installing
helm show values docker-selenium/selenium-grid > selenium-values.yaml

Create a values override file:

# my-values.yaml
global:
  seleniumGrid:
    imageTag: "4.18.1-20240224"
    nodesImageTag: "4.18.1-20240224"

hub:
  replicaCount: 1
  serviceType: ClusterIP
  resources:
    requests:
      memory: "512Mi"
      cpu: "250m"
    limits:
      memory: "1Gi"
      cpu: "1000m"

chromeNode:
  enabled: true
  replicas: 2
  maxSessions: 4
  sessionTimeout: 300
  resources:
    requests:
      memory: "1Gi"
      cpu: "500m"
    limits:
      memory: "3Gi"
      cpu: "2000m"

firefoxNode:
  enabled: true
  replicas: 1
  maxSessions: 2
  resources:
    requests:
      memory: "1Gi"
      cpu: "500m"
    limits:
      memory: "2Gi"
      cpu: "1500m"

edgeNode:
  enabled: false

Install:

helm install selenium-grid docker-selenium/selenium-grid \
  --namespace selenium \
  --create-namespace \
  --values my-values.yaml

# Verify
helm -n selenium list
kubectl -n selenium get pods

Horizontal Pod Autoscaler

The HPA scales Node replicas based on CPU utilization or custom metrics. For Selenium Grid, CPU-based scaling works reasonably well: active browser sessions drive CPU usage up, and the HPA responds by creating more Node pods.

# chrome-node-hpa.yaml
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: selenium-node-chrome-hpa
  namespace: selenium
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: selenium-node-chrome
  minReplicas: 1
  maxReplicas: 10
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 70
  behavior:
    scaleUp:
      stabilizationWindowSeconds: 30   # React quickly to demand
      policies:
        - type: Pods
          value: 3                      # Add up to 3 pods at once
          periodSeconds: 60
    scaleDown:
      stabilizationWindowSeconds: 300  # Wait 5 min before scaling down
      policies:
        - type: Pods
          value: 1                      # Remove 1 pod at a time
          periodSeconds: 120
kubectl apply -f chrome-node-hpa.yaml
kubectl -n selenium get hpa
# NAME                        REFERENCE                         TARGETS   MINPODS   MAXPODS   REPLICAS
# selenium-node-chrome-hpa   Deployment/selenium-node-chrome   45%/70%   1         10        2

For more precise scaling based on Grid session queue depth, implement a custom metric using the Grid 4 GraphQL API. A small metrics adapter queries /graphql every 30 seconds and publishes session queue depth as a custom metric:

# grid-metrics-adapter.py — simplified example
import requests
import time
from prometheus_client import start_http_server, Gauge

QUEUE_DEPTH = Gauge('selenium_grid_session_queue_size', 'Sessions waiting for a node')
ACTIVE_SESSIONS = Gauge('selenium_grid_active_sessions', 'Currently active sessions')

GRAPHQL_QUERY = """
{
  grid {
    sessionCount
    sessionQueueSize
  }
}
"""

def collect():
    try:
        response = requests.post(
            'http://selenium-hub:4444/graphql',
            json={'query': GRAPHQL_QUERY},
            timeout=5
        )
        data = response.json()['data']['grid']
        QUEUE_DEPTH.set(data['sessionQueueSize'])
        ACTIVE_SESSIONS.set(data['sessionCount'])
    except Exception as e:
        print(f"Collection error: {e}")

if __name__ == '__main__':
    start_http_server(8080)
    while True:
        collect()
        time.sleep(30)

With this metric exposed, use a type: External metric in the HPA for queue-depth-based scaling.

RBAC Configuration

If your Grid pods need to interact with the Kubernetes API (for example, a custom scaler that creates/deletes Node pods dynamically), configure RBAC:

# selenium-rbac.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
  name: selenium-grid
  namespace: selenium

---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: selenium-grid-role
  namespace: selenium
rules:
  - apiGroups: ["apps"]
    resources: ["deployments", "deployments/scale"]
    verbs: ["get", "list", "watch", "update", "patch"]
  - apiGroups: [""]
    resources: ["pods"]
    verbs: ["get", "list", "watch", "delete"]
  - apiGroups: [""]
    resources: ["pods/log"]
    verbs: ["get", "list"]

---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: selenium-grid-binding
  namespace: selenium
subjects:
  - kind: ServiceAccount
    name: selenium-grid
    namespace: selenium
roleRef:
  kind: Role
  name: selenium-grid-role
  apiGroup: rbac.authorization.k8s.io

Reference the service account in the Hub deployment:

spec:
  template:
    spec:
      serviceAccountName: selenium-grid
      containers:
        - name: selenium-hub

Ingress Configuration

For external access to the Grid UI and WebDriver endpoint, configure an Ingress. This assumes NGINX Ingress Controller is installed:

# selenium-ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: selenium-hub-ingress
  namespace: selenium
  annotations:
    nginx.ingress.kubernetes.io/proxy-read-timeout: "3600"
    nginx.ingress.kubernetes.io/proxy-send-timeout: "3600"
    nginx.ingress.kubernetes.io/proxy-connect-timeout: "3600"
    # WebSocket support for Grid's Event Bus
    nginx.ingress.kubernetes.io/proxy-http-version: "1.1"
    nginx.ingress.kubernetes.io/configuration-snippet: |
      proxy_set_header Upgrade $http_upgrade;
      proxy_set_header Connection "upgrade";
spec:
  ingressClassName: nginx
  rules:
    - host: selenium.internal.example.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: selenium-hub
                port:
                  number: 4444

The timeout annotations are critical: WebDriver sessions hold HTTP connections open for the duration of the test. Default NGINX timeouts (60s) will terminate active sessions mid-test.

Persistent Volumes for Video Recordings

If you enable video recording in Kubernetes, use a PersistentVolumeClaim backed by a ReadWriteMany storage class (NFS, Ceph, or cloud-provider equivalent) so multiple Node pods can write recordings:

# recordings-pvc.yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: selenium-recordings
  namespace: selenium
spec:
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 50Gi
  storageClassName: nfs-client  # Adjust to your storage class

Mount in the video sidecar container:

containers:
  - name: selenium-video
    image: selenium/video:ffmpeg-7.0.1-20240224
    volumeMounts:
      - name: recordings
        mountPath: /videos
    env:
      - name: DISPLAY_CONTAINER_NAME
        value: localhost   # sidecar shares pod network namespace
      - name: SE_VIDEO_FILE_NAME
        value: auto
volumes:
  - name: recordings
    persistentVolumeClaim:
      claimName: selenium-recordings

Verifying the Deployment

# Check all pods are running
kubectl -n selenium get pods -o wide

# Check Hub is ready
kubectl -n selenium exec -it deployment/selenium-hub -- \
  curl -s http://localhost:4444/status | python3 -m json.tool

# Check Node registration
kubectl -n selenium logs deployment/selenium-node-chrome | grep "Registered"
# INFO [NodeServer.lambda$createHandlers$5] - Registered with distributor. Continuing...

# Port-forward for local access
kubectl -n selenium port-forward service/selenium-hub 4444:4444 &
curl -s http://localhost:4444/status | python3 -c "
import sys, json
d = json.load(sys.stdin)
print('Ready:', d['value']['ready'])
print('Nodes:', len(d['value']['nodes']))
"

Troubleshooting Common Issues

Node pods start but don't register with Hub:

kubectl -n selenium logs deployment/selenium-node-chrome | tail -50
# Look for: "Unable to bind" or connection refused errors
# SE_EVENT_BUS_HOST must resolve — test with:
kubectl -n selenium exec -it deployment/selenium-node-chrome -- \
  nslookup selenium-hub

Sessions fail with "no nodes available":

# Check node slots via GraphQL
kubectl -n selenium exec -it deployment/selenium-hub -- \
  curl -s -X POST http://localhost:4444/graphql \
  -H "Content-Type: application/json" \
  -d '{"query": "{ nodesInfo { nodes { id availability slots { id session { id } } } } }"}'

OOMKilled on Node pods:

kubectl -n selenium describe pod <chrome-node-pod> | grep -A5 "OOM"
# Increase memory limit and verify /dev/shm is mounted:
kubectl -n selenium exec -it <chrome-node-pod> -- df -h /dev/shm

Hub pod restarts on high load:

The Hub process is lightweight (it doesn't run browsers), but under heavy session request load it can exceed memory limits. Increase the Hub's limit to at least 2Gi for large fleets:

resources:
  limits:
    memory: "2Gi"
    cpu: "2000m"

Kubernetes gives you the elasticity to run Grid at whatever scale your test suite demands. The combination of Deployments for baseline capacity, HPA for burst scaling, and proper resource limits means your test infrastructure can absorb a full regression run without manual intervention — and shrink back down overnight to minimize cloud costs.

Read more

Start now free