Hermione (Testplane): The Browser Testing Framework from Yandex
Hermione is a browser testing framework built by Yandex for running E2E and visual regression tests across multiple browsers in parallel. In 2023, Yandex rebranded it as Testplane. The framework is in production use at Yandex, Mail.ru, and other large Russian tech companies, and it's a practical alternative to Playwright or WebdriverIO for teams that need heavy parallelization and built-in visual testing.
What Testplane Offers
- Parallel browser execution — run tests across multiple browsers simultaneously, not sequentially
- Mocha-based test syntax — familiar
describe/itwith async/await - WebdriverIO commands — tests use WebdriverIO's API for browser interaction
- Built-in retry logic — automatic test retries on failure with configurable count
- Screenshot-based visual testing — compare screenshots to baseline with configurable tolerance
- Browser session reuse — option to reuse browser sessions across tests for speed
- Plugin system — extensible via plugins for reporting, visual comparison, and more
Installation
npm install --save-dev testplane
# or with yarn
yarn add --dev testplaneYou need a browser driver. For Chrome:
npm install --save-dev chromedriverOr use a Selenium Grid / Selenoid setup for multi-browser execution.
Basic Configuration
Create .testplane.conf.js in your project root:
module.exports = {
baseUrl: 'https://example.com',
gridUrl: 'http://localhost:4444/wd/hub',
browsers: {
chrome: {
desiredCapabilities: {
browserName: 'chrome',
chromeOptions: {
args: ['--headless', '--no-sandbox', '--disable-dev-shm-usage']
}
}
},
firefox: {
desiredCapabilities: {
browserName: 'firefox',
'moz:firefoxOptions': {
args: ['-headless']
}
}
}
},
sets: {
desktop: {
files: ['tests/desktop/**/*.test.js'],
browsers: ['chrome', 'firefox']
},
mobile: {
files: ['tests/mobile/**/*.test.js'],
browsers: ['chrome']
}
},
// Retry failed tests up to 2 times
retry: 2,
// Timeout per test in milliseconds
httpTimeout: 60000,
testTimeout: 90000,
// Number of parallel sessions per browser
sessionsPerBrowser: 5,
// Max total parallel sessions
parallelLimit: 10
};Writing Tests
Testplane tests use Mocha's describe/it structure with a browser object injected automatically:
describe('Product search', () => {
it('should display results for a valid query', async ({ browser }) => {
await browser.url('/search?q=headphones');
const searchResults = await browser.$('.search-results');
await searchResults.waitForDisplayed({ timeout: 5000 });
const resultCount = await browser.$$('.product-card');
assert.ok(resultCount.length > 0, 'Should have at least one result');
const firstResult = await browser.$('.product-card:first-child .product-name');
const productName = await firstResult.getText();
assert.ok(productName.length > 0, 'Product name should not be empty');
});
it('should show no-results message for gibberish query', async ({ browser }) => {
await browser.url('/search?q=xyzzy123gibberish');
const noResults = await browser.$('.no-results-message');
await noResults.waitForDisplayed({ timeout: 3000 });
const message = await noResults.getText();
assert.ok(
message.includes('No results'),
`Expected no-results message, got: ${message}`
);
});
});The browser object is WebdriverIO's browser API — commands like $(), $$(), waitForDisplayed(), getText(), setValue(), and click() all work as expected.
Setup and Teardown
Use before, beforeEach, after, and afterEach hooks:
describe('Authenticated features', () => {
before(async ({ browser }) => {
// Login once for all tests in this suite
await browser.url('/login');
await browser.$('#email').setValue('user@example.com');
await browser.$('#password').setValue('pass123');
await browser.$('#login-btn').click();
await browser.$('.dashboard').waitForDisplayed({ timeout: 5000 });
});
it('should show user profile', async ({ browser }) => {
await browser.url('/profile');
const name = await browser.$('.profile-name').getText();
assert.ok(name.length > 0);
});
it('should show user orders', async ({ browser }) => {
await browser.url('/orders');
const orderList = await browser.$('.order-list');
await orderList.waitForDisplayed();
// assertions...
});
});Visual Regression Testing
Testplane's killer feature is built-in screenshot comparison. Install the screenshot plugin:
npm install --save-dev testplane-screenshotsConfigure it:
module.exports = {
// ... other config ...
plugins: {
'testplane-screenshots': {
enabled: true,
screenshotsDir: './screenshots',
diffColor: '#ff0000',
tolerance: 2.3, // acceptable pixel difference percentage
antialiasingTolerance: 4
}
}
};In tests:
it('product card should match baseline', async ({ browser }) => {
await browser.url('/products/42');
const productCard = await browser.$('.product-detail-card');
await productCard.waitForDisplayed();
// Take screenshot and compare to baseline
await browser.assertView('product-detail', '.product-detail-card', {
ignoreElements: ['.last-updated-timestamp'] // ignore dynamic content
});
});On first run, screenshots are saved as baselines. Subsequent runs compare against baselines and fail if pixel differences exceed the tolerance threshold.
Update baselines after intentional UI changes:
npx testplane --update-refsParallel Execution
Testplane parallelizes across browsers and across test files. Configure concurrency:
module.exports = {
// ...
sessionsPerBrowser: 3, // 3 parallel Chrome sessions
parallelLimit: 10, // max 10 sessions total
browsers: {
chrome: { /* ... */ },
firefox: { /* ... */ }
}
};With sessionsPerBrowser: 3 and two browsers, Testplane runs up to 6 tests simultaneously. This scales well on CI machines with multiple CPUs.
Selenoid Integration
For multi-browser testing without installing browsers locally, use Selenoid:
# docker-compose.yml
version: '3'
services:
selenoid:
image: aerokube/selenoid:latest
ports:
- "4444:4444"
volumes:
- ./browsers.json:/etc/selenoid/browsers.json
- /var/run/docker.sock:/var/run/docker.sockConfigure Testplane to point at Selenoid:
module.exports = {
gridUrl: 'http://localhost:4444/wd/hub',
browsers: {
chrome: {
desiredCapabilities: {
browserName: 'chrome',
version: '120.0',
enableVNC: true, // for Selenoid UI
enableVideo: false
}
},
firefox: {
desiredCapabilities: {
browserName: 'firefox',
version: '120.0'
}
}
}
};Running Tests
# Run all tests
npx testplane
# Run specific test file
npx testplane tests/product-search.test.js
# Run in specific browser only
npx testplane --browser chrome
# Run specific set
npx testplane --set desktop
# Update visual baselines
npx testplane --update-refs
# GUI mode (interactive test runner)
npx testplane guiThe GUI mode opens a browser interface where you can run individual tests, see results, and compare visual diffs interactively.
CI Integration
# GitHub Actions
name: Testplane Tests
on: [push, pull_request]
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: '20'
- name: Install dependencies
run: npm ci
- name: Start Selenoid
run: |
docker-compose up -d selenoid
# Wait for Selenoid to be ready
npx wait-on http://localhost:4444/status
- name: Run Testplane tests
run: npx testplane
- name: Upload screenshots on failure
if: failure()
uses: actions/upload-artifact@v3
with:
name: testplane-screenshots
path: screenshots/Testplane vs Playwright
Both frameworks run browser tests. Key differences:
Testplane (Hermione):
- Uses WebdriverIO API (Selenium-based)
- Better Selenoid integration for multi-browser grids
- Built-in visual regression with screenshot comparison
- Mature retry and parallelization configuration
- More common in Russian-speaking tech ecosystems
Playwright:
- Uses Playwright's own browser automation (faster)
- Built-in multi-browser support without Selenium Grid
- Better cross-browser isolation out of the box
- Larger English-language community and documentation
For projects already using Selenium Grid or Selenoid infrastructure, Testplane integrates well. For new projects without existing Selenium infrastructure, Playwright typically has lower setup overhead.
Production Monitoring
Testplane tests run during CI — they verify behavior at deployment time. For production confidence between deployments, HelpMeTest monitors your critical user flows continuously. No WebDriver, no Selenoid, no Node.js build pipeline required — just plain-English test descriptions running on a schedule.
Summary
Testplane (formerly Hermione) is a production-grade browser testing framework optimized for parallelization and visual regression. If you're running a large suite across multiple browsers and need built-in screenshot comparison, it's worth considering alongside Playwright and WebdriverIO. The WebdriverIO command API means existing Selenium knowledge transfers directly, and the Selenoid integration is first-class.