Test Automation

Golden Screenshot Testing in Flutter

HelpMeTest

HelpMeTest

— 6 min read
Golden Screenshot Testing in Flutter

Golden tests (also called screenshot tests or snapshot tests) capture a widget's rendered output as a PNG and compare it pixel-by-pixel against a stored reference image. They are extraordinarily effective at catching unintended visual regressions — a padding change, a wrong color, a misaligned text node — that functional tests completely miss.

Flutter ships with matchesGoldenFile built in. The ecosystem extends this with golden_toolkit and alchemist, which solve the real-world pain points: font loading, multi-device captures, and managing golden updates on CI. This post covers the full picture.

The Basic Mechanism

import 'package:flutter/material.dart';
import 'package:flutter_test/flutter_test.dart';

void main() {
  testWidgets('UserCard matches golden', (tester) async {
    await tester.pumpWidget(
      MaterialApp(
        home: Scaffold(
          body: UserCard(
            name: 'Alice Johnson',
            role: 'Senior Engineer',
            avatarUrl: null,
          ),
        ),
      ),
    );

    await expectLater(
      find.byType(UserCard),
      matchesGoldenFile('goldens/user_card.png'),
    );
  });
}

On first run (or when no golden exists), Flutter writes the current render to goldens/user_card.png. On subsequent runs, it renders the widget fresh and compares it to the stored PNG. Any pixel difference fails the test.

Golden files are checked into source control. They live next to your test files:

test/
  widgets/
    user_card_test.dart
    goldens/
      user_card.png

Updating Goldens

When you intentionally change a widget's appearance, the golden becomes stale. Update it:

flutter test --update-goldens

This regenerates all golden files. Review the diffs in git before committing — the visual diff should match your intent. Many teams use git diff --stat to spot unexpected golden changes in PRs.

To update a single test's golden:

flutter test test/widgets/user_card_test.dart --update-goldens

The Font Problem

The most common frustration with golden tests is font rendering. Flutter test environment does not load fonts by default. Text renders as boxes:

████████ ████ ████████

This makes goldens that include any text nearly useless without font loading. Fix it by loading fonts in your test setup:

testWidgets('UserCard with real fonts', (tester) async {
  // Load fonts from pubspec.yaml assets
  final fonts = await loadAppFonts(); // from golden_toolkit
  addTearDown(fonts.dispose);

  await tester.pumpWidget(/* ... */);
  await expectLater(find.byType(UserCard), matchesGoldenFile('goldens/user_card.png'));
});

golden_toolkit provides loadAppFonts() which reads your pubspec.yaml and loads all declared fonts automatically.

golden_toolkit: The Production-Ready Solution

Add golden_toolkit to your dev dependencies:

dev_dependencies:
  golden_toolkit: ^0.15.0
  flutter_test:
    sdk: flutter

GoldenToolkit Setup

Create a flutter_test_config.dart at the root of your test/ directory. Flutter test runner auto-executes this file before any test:

// test/flutter_test_config.dart
import 'dart:async';
import 'package:golden_toolkit/golden_toolkit.dart';

Future<void> testExecutable(FutureOr<void> Function() testMain) async {
  return GoldenToolkit.runWithConfiguration(
    testMain,
    config: GoldenToolkitConfiguration(
      skipGoldenAssertion: () => !Platform.isMacOS, // Only compare on macOS
      enableRealShadows: true,
    ),
  );
}

The skipGoldenAssertion callback is important: font rendering and anti-aliasing differ between Linux (CI) and macOS (developer machines). More on this below.

loadAppFonts and pumpWidgetBuilder

import 'package:golden_toolkit/golden_toolkit.dart';

void main() {
  setUp(() async {
    await loadAppFonts();
  });

  testWidgets('Button golden', (tester) async {
    await tester.pumpWidgetBuilder(
      const PrimaryButton(label: 'Get Started'),
      surfaceSize: const Size(300, 80),
    );

    await screenMatchesGolden(tester, 'primary_button');
  });
}

pumpWidgetBuilder wraps your widget in a MaterialApp with proper theme, and crops the golden to the exact widget bounds (using surfaceSize). No need to render the entire screen when you just want the component.

screenMatchesGolden is golden_toolkit's replacement for matchesGoldenFile. It stores goldens in a goldens/ subdirectory automatically and names them after your test.

Multi-Device Goldens

Test the same widget across multiple screen sizes in one shot:

void main() {
  setUp(() async => loadAppFonts());

  testWidgets('ResponsiveCard on multiple devices', (tester) async {
    await tester.pumpWidgetBuilder(const ResponsiveCard(title: 'Hello'));

    await multiScreenGolden(
      tester,
      'responsive_card',
      devices: [
        Device.phone,
        Device.tabletPortrait,
        Device.tabletLandscape,
        const Device(
          name: 'large_phone',
          size: Size(428, 926),
          devicePixelRatio: 3.0,
        ),
      ],
    );
  });
}

This generates separate golden files for each device:

  • goldens/responsive_card.phone.png
  • goldens/responsive_card.tabletPortrait.png
  • goldens/responsive_card.tabletLandscape.png
  • goldens/responsive_card.large_phone.png

Custom Pump Function

For widgets that need async setup (loading data, animations settling):

testWidgets('AnimatedCounter after animation', (tester) async {
  await tester.pumpWidgetBuilder(
    const AnimatedCounter(value: 42),
    // Custom pump to let animation complete
    wrapper: materialAppWrapper(theme: AppTheme.light),
  );

  // Advance animation
  await tester.pump(const Duration(milliseconds: 500));

  await screenMatchesGolden(tester, 'animated_counter_final');
});

Handling Platform Rendering Differences

The hardest problem in golden testing is that pixel output differs between operating systems. The same widget looks subtly different on macOS vs Linux because:

  1. Font rasterization differs between CoreText (macOS) and FreeType (Linux).
  2. Anti-aliasing algorithms differ.
  3. Shadow rendering can vary.

Strategies for managing this:

Strategy 1: Only Compare on One Platform

The simplest: generate and compare goldens only on macOS (or only on Linux CI). Skip on other platforms:

// In flutter_test_config.dart
config: GoldenToolkitConfiguration(
  skipGoldenAssertion: () => !Platform.isMacOS,
),

Developers on macOS update goldens locally. CI on Linux skips the pixel comparison (but still renders widgets, catching layout errors).

Strategy 2: Machine-Specific Goldens

Store separate goldens per platform:

String goldenPath(String name) {
  if (Platform.isMacOS) return 'goldens/mac/$name.png';
  if (Platform.isLinux) return 'goldens/linux/$name.png';
  return 'goldens/$name.png';
}

await expectLater(
  find.byType(MyWidget),
  matchesGoldenFile(goldenPath('my_widget')),
);

CI and developer machines maintain separate golden directories. This doubles storage but eliminates false failures.

Strategy 3: Tolerance Thresholds

Accept small pixel differences using a custom comparator. Flutter lets you override the global comparator:

// test/flutter_test_config.dart
void main() {
  goldenFileComparator = TolerantComparator(
    Uri.parse('test/'),
    threshold: 0.005, // 0.5% pixel difference tolerance
  );
}

Implementing TolerantComparator:

class TolerantComparator extends LocalFileComparator {
  final double threshold;

  TolerantComparator(Uri testFile, {this.threshold = 0.005})
    : super(testFile);

  @override
  Future<bool> compare(Uint8List imageBytes, Uri golden) async {
    final result = await GoldenFileComparator.compareLists(
      imageBytes,
      await getGoldenBytes(golden),
    );

    if (!result.passed && result.diffPercent <= threshold) {
      // Within tolerance — consider it passing
      return true;
    }

    return result.passed;
  }
}

Strategy 4: Use alchemist (Canvaskit-based)

The alchemist package renders widgets using the Canvaskit renderer, which produces identical output across platforms. It is the most robust solution for cross-platform consistency:

dev_dependencies:
  alchemist: ^0.7.0
import 'package:alchemist/alchemist.dart';

void main() {
  goldenTest(
    'UserCard renders correctly',
    fileName: 'user_card',
    builder: () => GoldenTestGroup(
      children: [
        GoldenTestScenario(
          name: 'default',
          child: const UserCard(name: 'Alice', role: 'Engineer'),
        ),
        GoldenTestScenario(
          name: 'long name',
          child: const UserCard(
            name: 'Alexandria Bartholomew-Whitmore',
            role: 'Principal Software Engineer',
          ),
        ),
      ],
    ),
  );
}

Alchemist renders all scenarios on a single golden image with labels, making visual review much easier.

What Makes a Good Golden Test

Golden tests are most valuable for:

  • Design system components — Buttons, cards, badges, chips. These are shared widely and regressions ripple everywhere.
  • Complex layouts — Grid views, data tables, multi-column layouts where spacing bugs are easy to introduce.
  • Themed variants — Same widget in light mode, dark mode, high-contrast.
  • State variations — Loading, error, empty, populated — each captured separately.
  • Typography-heavy widgets — Golden tests catch line height and spacing bugs that functional tests miss.

Avoid golden tests for:

  • Highly animated widgets (capture at a specific frame instead).
  • Widgets whose content changes frequently (like timestamps).
  • Platform-specific UI (date pickers, native switches).

Organizing Golden Files in the Repository

test/
  goldens/            ← Generated files, committed to git
    mac/              ← macOS-specific (if using per-platform strategy)
      user_card.png
      primary_button.png
    linux/
      user_card.png
      primary_button.png
  widgets/
    user_card_test.dart
    button_test.dart
  flutter_test_config.dart

Add a .gitattributes entry to handle PNG binary diffs cleanly:

test/goldens/**/*.png binary

This prevents Git from trying to text-diff PNG files.

CI Integration

# .github/workflows/golden_tests.yml
name: Golden Tests

on: [pull_request]

jobs:
  golden:
    runs-on: ubuntu-latest  # or macos-14 if you use macOS goldens

    steps:
      - uses: actions/checkout@v4

      - uses: subosito/flutter-action@v2
        with:
          flutter-version: '3.22.0'

      - name: Install dependencies
        run: flutter pub get

      - name: Run golden tests
        run: flutter test test/widgets/ --tags golden

      - name: Upload golden diff artifacts on failure
        if: failure()
        uses: actions/upload-artifact@v4
        with:
          name: golden-failures
          path: test/failures/   # golden_toolkit writes diffs here
          retention-days: 7

When a golden test fails, golden_toolkit writes three images to test/failures/:

  • *_masterImage.png — the stored golden
  • *_testImage.png — what was actually rendered
  • *_isolatedDiff.png — pixels that differ (highlighted)

Uploading these as CI artifacts lets reviewers see exactly what changed without checking out the branch.

Tagging Golden Tests

Tag golden tests so you can run them separately from fast unit tests:

testWidgets(
  'UserCard golden',
  tags: ['golden'],
  (tester) async {
    // ...
  },
);
flutter test --tags golden          # Only golden tests
flutter test --exclude-tags golden  # Everything except golden tests

This lets CI run fast tests on every push and golden tests only on PRs or scheduled runs.

Conclusion

Golden tests occupy a unique position in the Flutter testing hierarchy: they cannot be replaced by functional tests, and they catch entire categories of visual regression automatically. The setup friction — font loading, platform consistency, CI configuration — is real but manageable with golden_toolkit or alchemist. Start with your design system components (they have the highest regression risk and the most stable expected output), establish a per-platform or tolerance-based comparison strategy, and wire golden failures to upload diff images in CI. Once in place, golden tests silently protect your UI and surface any unintended change before it reaches a user.

Read more

Start now free