Testing

Elixir Mix Test: Configuration, Coverage, Tags & CI Setup

25 Sep 2026 — 5 min read

mix test is the entry point for all Elixir testing. While ExUnit handles test structure and assertions, mix test controls how tests are discovered, filtered, and executed. This guide covers the configuration knobs that matter most in production Elixir projects — from test tagging to coverage reporting to CI pipeline setup.

Basic Mix Test Configuration

Your project's mix.exs controls test behavior through the :test key and ExUnit options:

# mix.exs
def project do
  [
    app: :my_app,
    version: "0.1.0",
    elixir: "~> 1.15",
    elixirc_paths: elixirc_paths(Mix.env()),
    start_permanent: Mix.env() == :prod,
    deps: deps(),
    test_coverage: [tool: ExCoveralls],
    preferred_cli_env: [
      coveralls: :test,
      "coveralls.detail": :test,
      "coveralls.post": :test,
      "coveralls.html": :test
    ]
  ]
end

defp elixirc_paths(:test), do: ["lib", "test/support"]
defp elixirc_paths(_), do: ["lib"]

The elixirc_paths/1 pattern is essential — it makes your test/support helpers available during tests without polluting production builds.

Test Tags and Filtering

Tags let you categorize tests and run subsets selectively. This is critical for large test suites where running everything on every change is too slow.

defmodule MyApp.SlowTest do
  use ExUnit.Case, async: false

  @moduletag :integration
  @moduletag timeout: 120_000

  @tag :slow
  test "imports large dataset" do
    # Takes 30+ seconds
    assert {:ok, count} = MyApp.Importer.run("large_fixture.csv")
    assert count > 10_000
  end

  @tag :external
  @tag :slow
  test "calls third-party API" do
    assert {:ok, _response} = MyApp.ExternalAPI.fetch_data()
  end
end

Run or exclude by tag:

# Run only integration tests
mix test --only integration

# Run only slow tests
mix test --only slow

# Exclude slow and external tests (fast local dev loop)
mix test --exclude slow --exclude external

# Run a single tagged test
mix test --only "test calls third-party"

Configure default exclusions in test/test_helper.exs so developers never accidentally run slow tests locally:

# test/test_helper.exs
ExUnit.start()
ExUnit.configure(exclude: [:integration, :external])

With this setup, mix test is fast by default. CI explicitly includes all tags:

# CI command — runs everything
mix test --include integration --include external

Async Tests

async: true runs test modules concurrently. This is the single biggest performance lever in ExUnit:

# Safe for async — no shared mutable state
defmodule MyApp.PureLogicTest do
  use ExUnit.Case, async: true

  test "calculates discount" do
    assert MyApp.Pricing.discount(:vip, 100.0) == 20.0
  end
end

# NOT safe for async — touches shared database or ETS
defmodule MyApp.UserRepoTest do
  use ExUnit.Case, async: false  # explicit

  setup do
    :ok = Ecto.Adapters.SQL.Sandbox.checkout(MyApp.Repo)
  end
end

Rule of thumb: pure functions and process-isolated tests → async: true. Anything touching the database, ETS, or global config → async: false.

When using Ecto's SQL sandbox, async is actually safe if you use sandbox mode correctly:

# test/test_helper.exs
Ecto.Adapters.SQL.Sandbox.mode(MyApp.Repo, :manual)

# In each test module with DB access
defmodule MyApp.UserRepoTest do
  use ExUnit.Case, async: true  # Safe with sandbox!

  setup do
    :ok = Ecto.Adapters.SQL.Sandbox.checkout(MyApp.Repo)
  end
end

Test Coverage with ExCoveralls

Add ExCoveralls to your deps:

# mix.exs
defp deps do
  [
    {:excoveralls, "~> 0.18", only: :test}
  ]
end

Run coverage:

# Terminal summary
mix coveralls

# Detailed line-by-line report
mix coveralls.detail

# HTML report in cover/excoveralls.html
mix coveralls.html

# JSON for CI systems
mix coveralls.json

Configure coverage thresholds and ignore paths in coveralls.json at your project root:

{
  "coverage_options": {
    "minimum_coverage": 80,
    "treat_no_relevant_lines_as_covered": true
  },
  "skip_files": [
    "lib/my_app_web/telemetry.ex",
    "lib/my_app/release.ex",
    "test/"
  ]
}

The minimum_coverage threshold causes mix coveralls to exit with a non-zero code if coverage drops below 80%, which makes CI fail fast.

Running Specific Tests

Several ways to run a subset:

# Run a single file
mix test test/my_app/user_test.exs

# Run tests at a specific line
mix test test/my_app/user_test.exs:42

# Run tests matching a pattern (partial name)
mix test --grep "returns error"

# Run failed tests from last run (requires --failed flag)
mix test --failed

The --failed flag is a significant productivity boost — after a failing run, you iterate on only the broken tests without re-running the entire suite.

Parallel Test Execution Across Files

ExUnit already runs test modules concurrently when async: true is set. But for large monorepos with umbrella apps, you can parallelize across applications:

# In an umbrella app
cd apps/my_core && mix test &
cd apps/my_web && mix test &
wait

Or use a script:

#!/bin/bash
set -e

pids=()
for app in apps/*/; do
  (cd "$app" && mix test) &
  pids+=($!)
done

for pid in "${pids[@]}"; do
  wait "$pid"
done

Test Output and Formatting

Control output verbosity:

# Verbose — shows each test name as it runs
mix test --trace

# Seed — reproduce a specific random order
mix test --seed 42

# Max failures — stop after N failures
mix test --max-failures 5

# Slowest tests — identify bottlenecks
mix test --slowest 10

The --trace flag is invaluable when debugging deadlocks or timeouts — you can see exactly which test is hanging.

Doctests

Elixir's doctest feature runs code examples in @doc attributes as tests:

defmodule MyApp.Math do
  @doc """
  Adds two numbers.

      iex> MyApp.Math.add(1, 2)
      3

      iex> MyApp.Math.add(-1, 1)
      0
  """
  def add(a, b), do: a + b
end

defmodule MyApp.MathTest do
  use ExUnit.Case

  doctest MyApp.Math
end

Run doctests as part of your normal suite — they double as living documentation. When the implementation drifts from the docs, tests fail.

CI Configuration

A complete GitHub Actions workflow for Elixir testing:

# .github/workflows/test.yml
name: Test

on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest

    services:
      postgres:
        image: postgres:15
        env:
          POSTGRES_PASSWORD: postgres
        options: >-
          --health-cmd pg_isready
          --health-interval 10s
          --health-timeout 5s
          --health-retries 5
        ports:
          - 5432:5432

    env:
      MIX_ENV: test
      DATABASE_URL: postgresql://postgres:postgres@localhost/my_app_test

    steps:
      - uses: actions/checkout@v4

      - uses: erlef/setup-beam@v1
        with:
          otp-version: "26"
          elixir-version: "1.15"

      - name: Restore deps cache
        uses: actions/cache@v3
        with:
          path: |
            deps
            _build
          key: ${{ runner.os }}-mix-${{ hashFiles('**/mix.lock') }}

      - name: Install deps
        run: mix deps.get

      - name: Compile
        run: mix compile --warnings-as-errors

      - name: Run tests with coverage
        run: mix coveralls.json --include integration

      - name: Upload coverage
        uses: codecov/codecov-action@v3
        with:
          file: ./cover/excoveralls.json

Seed-Based Reproducibility

ExUnit randomizes test order by default to expose order-dependent failures. Always capture the seed from a failed CI run:

Randomized with seed 98765

Then reproduce locally:

mix test --seed 98765

If a test only fails in certain orders, you have a test isolation problem — global state is leaking between tests.

Watching Tests During Development

The mix test.watch package gives you file-watching test re-runs:

# mix.exs deps
{:mix_test_watch, "~> 1.0", only: [:dev, :test], runtime: false}

mix test.watch

Now tests re-run automatically when you save a file. Combine with --stagger-test-time-out to avoid database connection pool exhaustion during rapid re-runs.

Summary

Effective mix test configuration follows a simple hierarchy: fast by default, comprehensive in CI. Use tags to exclude slow/external tests from the default run, enable async: true wherever safe, set a coverage minimum to catch regressions, and use --failed to iterate quickly when fixing broken tests. These four habits keep test cycles short and confidence high.

For teams running HelpMeTest alongside their Elixir unit tests, the combination covers different risk surfaces: ExUnit verifies internal logic in isolation while HelpMeTest monitors live application behavior end-to-end, catching integration failures that unit tests structurally cannot detect.