Selenium

Selenium Grid 4 Setup Guide: Configure Your Distributed Testing Infrastructure

HelpMeTest

HelpMeTest

— 8 min read
Selenium Grid 4 Setup Guide: Configure Your Distributed Testing Infrastructure

Selenium Grid 4 represents a complete architectural overhaul from its predecessors, replacing the monolithic Hub/Node model with a microservices-oriented design that scales cleanly from a single laptop to a multi-datacenter fleet. If you've been putting off migrating from Grid 3 or standing up a fresh Grid environment, this guide walks you through everything from first download to running tests across multiple browsers simultaneously.

Understanding Grid 4 Architecture

Before installing anything, it pays to understand what you're actually deploying. Grid 4 decomposes what was previously a single Hub process into distinct components, each with a well-defined responsibility.

Router — the front door. All incoming WebDriver requests land here first. The Router decides whether a request belongs to an existing session (and forwards it to the right Node) or needs a new session (and hands it to the Distributor).

Distributor — the matchmaker. When a new session request arrives, the Distributor looks at all registered Nodes, finds one that has capacity and matches the requested capabilities, and assigns the session. It maintains a queue when all Nodes are busy.

Session Map — the directory. This component tracks which session ID lives on which Node. The Router consults the Session Map on every forwarded request.

Node — the worker. Each Node controls one or more browser instances. Nodes register themselves with the Distributor at startup and report their available slots (e.g., "I can run 4 Chrome sessions and 2 Firefox sessions simultaneously").

Event Bus — the nervous system. Components communicate asynchronously via the Event Bus rather than calling each other directly. This decoupling is what allows fully distributed deployments.

Grid 4 ships three deployment modes that compose these components differently:

  • Standalone: All components run in a single JVM process. Perfect for local development and CI runners where you control the whole machine.
  • Hub and Node: Hub bundles Router, Distributor, Session Map, and Event Bus. Nodes remain separate. This mirrors Grid 3's topology and is the most common production setup.
  • Fully Distributed: Every component runs as its own process (or container). Used when you need independent scaling of, say, the Distributor without scaling the Router.

Prerequisites

Grid 4 requires Java 11 or later. Verify your version:

java -version
# openjdk version "17.0.9" 2023-10-17

Download the Grid jar from the official releases page. As of this writing, 4.18.x is the latest stable series:

wget https://github.com/SeleniumHQ/selenium/releases/download/selenium-4.18.1/selenium-server-4.18.1.jar

You'll also need browser drivers on the PATH of every machine that will run a Node. Grid 4 supports Selenium Manager for automatic driver management, but for production deployments you generally want pinned versions:

# ChromeDriver — match your Chrome version
wget https://chromedriver.storage.googleapis.com/120.0.6099.109/chromedriver_linux64.zip
unzip chromedriver_linux64.zip
chmod +x chromedriver
sudo mv chromedriver /usr/local/bin/

# GeckoDriver for Firefox
wget https://github.com/mozilla/geckodriver/releases/download/v0.34.0/geckodriver-v0.34.0-linux64.tar.gz
tar -xzf geckodriver-v0.34.0-linux64.tar.gz
sudo mv geckodriver /usr/local/bin/

Standalone Mode — Your First Grid

Standalone is the fastest path to a working Grid. One command, one process, zero coordination overhead:

java -jar selenium-server-4.18.1.jar standalone \
  --port 4444 \
  --session-request-timeout 300 \
  --session-retry-interval 5

The Grid UI is immediately available at http://localhost:4444. You'll see the console showing registered browsers and current session counts. Standalone automatically detects installed browsers using Selenium Manager.

To run a test against standalone Grid, point your RemoteWebDriver at port 4444:

from selenium import webdriver
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities

options = webdriver.ChromeOptions()
options.add_argument("--no-sandbox")
options.add_argument("--disable-dev-shm-usage")

driver = webdriver.Remote(
    command_executor="http://localhost:4444",
    options=options
)

driver.get("https://example.com")
print(driver.title)
driver.quit()
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeOptions;
import org.openqa.selenium.remote.RemoteWebDriver;
import java.net.URL;

public class GridStandaloneTest {
    public static void main(String[] args) throws Exception {
        ChromeOptions options = new ChromeOptions();
        options.addArguments("--no-sandbox", "--disable-dev-shm-usage");

        WebDriver driver = new RemoteWebDriver(
            new URL("http://localhost:4444"),
            options
        );

        driver.get("https://example.com");
        System.out.println(driver.getTitle());
        driver.quit();
    }
}

Hub and Node Mode

Hub and Node is the right choice when you have a central coordination machine and multiple worker machines (physical, virtual, or containerized). Start the Hub first:

# On the hub machine (192.168.1.10)
java -jar selenium-server-4.18.1.jar hub \
  --port 4444 \
  --publish-events tcp://192.168.1.10:4442 \
  --subscribe-events tcp://192.168.1.10:4443

Then start Nodes on each worker machine, pointing them at the Hub:

# On worker machine 1 (192.168.1.20) — Chrome + Firefox
java -jar selenium-server-4.18.1.jar node \
  --hub http://192.168.1.10:4444 \
  --port 5555 \
  --max-sessions 6 \
  --override-max-sessions true

# On worker machine 2 (192.168.1.21) — Chrome only
java -jar selenium-server-4.18.1.jar node \
  --hub http://192.168.1.10:4444 \
  --port 5555 \
  --driver-configuration display-name=Chrome max-sessions=8 webdriver-path=/usr/local/bin/chromedriver

TOML Configuration Files

For anything beyond quick experiments, use TOML config files. They're more maintainable and easier to version-control than long argument lists:

# hub.toml
[server]
port = 4444
max-threads = 24

[events]
publish = "tcp://0.0.0.0:4442"
subscribe = "tcp://0.0.0.0:4443"

[distributor]
slot-matcher = "org.openqa.selenium.grid.data.DefaultSlotMatcher"
session-request-timeout = 300
session-retry-interval = 15
# node.toml
[server]
port = 5555
max-threads = 12

[node]
max-sessions = 8
session-timeout = 300
override-max-sessions = true

[[node.driver-configuration]]
display-name = "Chrome"
stereotype = '{"browserName": "chrome", "browserVersion": "120", "platformName": "linux"}'
max-sessions = 6
webdriver-path = "/usr/local/bin/chromedriver"

[[node.driver-configuration]]
display-name = "Firefox"
stereotype = '{"browserName": "firefox", "browserVersion": "121", "platformName": "linux"}'
max-sessions = 2
webdriver-path = "/usr/local/bin/geckodriver"

Launch with the config file:

java -jar selenium-server-4.18.1.jar hub --config hub.toml
java -jar selenium-server-4.18.1.jar node --config node.toml

Fully Distributed Mode

Fully distributed gives you independent process control over every component. This is typically used in Kubernetes where you want the Event Bus and Session Map as dedicated StatefulSets while the Distributor and Router scale horizontally.

# 1. Start the Event Bus
java -jar selenium-server-4.18.1.jar event-bus \
  --publish-events tcp://0.0.0.0:4442 \
  --subscribe-events tcp://0.0.0.0:4443 \
  --port 5557

# 2. Start the Session Map
java -jar selenium-server-4.18.1.jar sessions \
  --publish-events tcp://eventbus:4442 \
  --subscribe-events tcp://eventbus:4443 \
  --port 5556

# 3. Start the Session Queue
java -jar selenium-server-4.18.1.jar sessionqueue \
  --publish-events tcp://eventbus:4442 \
  --subscribe-events tcp://eventbus:4443 \
  --port 5559

# 4. Start the Distributor
java -jar selenium-server-4.18.1.jar distributor \
  --publish-events tcp://eventbus:4442 \
  --subscribe-events tcp://eventbus:4443 \
  --sessions http://sessions:5556 \
  --sessionqueue http://sessionqueue:5559 \
  --port 5553 \
  --bind-bus false

# 5. Start the Router
java -jar selenium-server-4.18.1.jar router \
  --sessions http://sessions:5556 \
  --distributor http://distributor:5553 \
  --sessionqueue http://sessionqueue:5559 \
  --port 4444 \
  --bind-bus false

# 6. Start Nodes (same as Hub-Node mode, just point to the router)
java -jar selenium-server-4.18.1.jar node \
  --publish-events tcp://eventbus:4442 \
  --subscribe-events tcp://eventbus:4443 \
  --port 5555

Configuring Browser Slots and Concurrency

One of the most impactful tuning decisions is how many concurrent sessions each Node runs. The default is conservative — Grid 4 caps sessions based on available CPU cores.

To override:

java -jar selenium-server-4.18.1.jar node \
  --max-sessions 8 \
  --override-max-sessions true \
  --session-timeout 300

For CI environments where browsers are ephemeral and headless, you can safely push beyond the CPU core count. A 4-core machine running headless Chrome can typically handle 6-8 concurrent sessions before queue times rise significantly.

Session Timeout Configuration

Sessions that hang or crash without explicit cleanup will lock up slots. Configure aggressive timeouts:

[node]
session-timeout = 300        # seconds before idle session is killed
max-sessions = 8
drain-after-session-count = 0  # set >0 to recycle node after N sessions

Verifying the Grid

The Grid exposes a /status endpoint that returns JSON describing every registered Node and its current capacity:

curl -s http://localhost:4444/status | python3 -m json.tool
{
  "value": {
    "ready": true,
    "message": "Selenium Grid ready.",
    "nodes": [
      {
        "id": "b9e5a0d3-...",
        "uri": "http://192.168.1.20:5555",
        "maxSessions": 8,
        "osInfo": {"arch": "amd64", "name": "Linux", "version": "5.15.0"},
        "heartbeatPeriod": 60000,
        "availability": "UP",
        "version": "4.18.1",
        "slots": [
          {
            "id": {"hostId": "b9e5a0d3-...", "id": "chrome-slot-1"},
            "lastStarted": "1970-01-01T00:00:00Z",
            "session": null,
            "stereotype": {"browserName": "chrome", "browserVersion": "120"}
          }
        ]
      }
    ]
  }
}

The GraphQL endpoint at /graphql provides richer queries for session queue depth, slot utilization, and node health — covered in detail in the monitoring guide.

Running Tests in Parallel

With Grid running, parallel execution is controlled at the test framework level, not by Grid itself. Grid simply fulfills concurrent session requests.

TestNG parallel configuration:

<!-- testng.xml -->
<suite name="GridSuite" parallel="tests" thread-count="6">
  <test name="ChromeTests">
    <parameter name="browser" value="chrome"/>
    <classes>
      <class name="com.example.tests.LoginTest"/>
      <class name="com.example.tests.CheckoutTest"/>
    </classes>
  </test>
  <test name="FirefoxTests">
    <parameter name="browser" value="firefox"/>
    <classes>
      <class name="com.example.tests.LoginTest"/>
      <class name="com.example.tests.CheckoutTest"/>
    </classes>
  </test>
</suite>

pytest-xdist with Grid:

# conftest.py
import pytest
from selenium import webdriver

def pytest_addoption(parser):
    parser.addoption("--browser", default="chrome")
    parser.addoption("--grid-url", default="http://localhost:4444")

@pytest.fixture
def driver(request):
    browser = request.config.getoption("--browser")
    grid_url = request.config.getoption("--grid-url")

    if browser == "chrome":
        options = webdriver.ChromeOptions()
        options.add_argument("--no-sandbox")
    elif browser == "firefox":
        options = webdriver.FirefoxOptions()

    driver = webdriver.Remote(command_executor=grid_url, options=options)
    yield driver
    driver.quit()
# Run 6 tests in parallel across Grid
pytest tests/ -n 6 --grid-url http://grid:4444

Firewall and Network Considerations

In Hub-Node deployments, several ports need to be open:

Port Component Direction
4444 Hub/Router Inbound from test runners
4442 Event Bus publish Node → Hub
4443 Event Bus subscribe Hub → Node
5555 Node Hub → Node (session forwarding)

If Nodes are behind NAT, set the external advertised address explicitly:

java -jar selenium-server-4.18.1.jar node \
  --hub http://hub:4444 \
  --host 0.0.0.0 \
  --port 5555 \
  --external-url http://192.168.1.20:5555

Common Pitfalls

Node registers but sessions fail immediately: Usually a driver version mismatch. The Node reports Chrome 120 capability but chromedriver is version 119. Always keep browser and driver versions in sync.

Sessions queue indefinitely: Either all slots are occupied or no Node matches the requested capabilities. Check the /status endpoint — if ready: false, the Distributor has no available Nodes. If ready but sessions queue, the requested browserVersion or platformName doesn't match any Node stereotype.

Hub unreachable from Nodes: Firewall blocking port 4442/4443. The Node starts, attempts to publish its registration event to the Event Bus, and silently fails. Enable debug logging to diagnose: --log-level FINE.

Memory pressure: Each browser instance consumes 200-400MB RAM. A Node with 8 Chrome slots needs at least 4GB free RAM. Monitor with free -h and set -Xmx on the JVM: java -Xmx2g -jar selenium-server-4.18.1.jar node ....

Next Steps

With Grid 4 running, the natural next steps are containerization (Docker Compose eliminates the manual Node setup entirely) and observability (Grid 4's GraphQL API and Prometheus endpoint give you real-time visibility into queue depth and session utilization). Both topics are covered in depth in subsequent posts in this series. The architecture you've set up here — whether standalone, hub-node, or fully distributed — is the foundation everything else builds on.

Read more

Start now free