Testing

Recce: dbt Change Testing and PR Review for Data Pipelines

HelpMeTest

HelpMeTest

— 5 min read
Recce: dbt Change Testing and PR Review for Data Pipelines

Recce is a dbt change review tool that answers one question: what did this PR actually change in the data? When you modify a dbt model, your CI runs dbt tests and they pass — but that only tells you the data is valid, not whether it changed. Recce builds the old and new versions of your models and lets you compare them row-by-row, column-by-column.

This is the data equivalent of a code diff: before merging, you see exactly what rows and values changed.

The Problem Recce Solves

Standard dbt CI:

  1. Run dbt build on the PR branch
  2. Run dbt tests
  3. If tests pass, merge

What this misses: logic changes that produce different (but valid) data. A filter change that drops 1,000 rows. A join type change from LEFT to INNER that changes aggregations. A date truncation bug that shifts revenue from December to January.

These changes pass all static tests but produce wrong analytics.

Recce adds:

  1. Build the base branch (main) in a separate schema
  2. Build the target branch (PR) in another schema
  3. Compare the outputs — show diffs at the row and column level

Installation

pip install recce

You need two dbt artifacts (manifest + catalog) — one from your base branch and one from your target branch. The simplest setup runs both dbt builds as part of CI.

Local Usage

Setup

Point Recce at your two environments. In profiles.yml:

my_project:
  target: dev
  outputs:
    dev:
      type: bigquery
      project: my-analytics
      dataset: dbt_myname
      # ...
    dev_base:
      type: bigquery
      project: my-analytics
      dataset: dbt_myname_base   # where main branch data lives
      # ...

Generate Base Artifacts

# On the base branch (main)
git stash  # or use a worktree
dbt build --target dev_base
dbt docs generate --target dev_base

This populates dev_base schema with current main data and generates manifest.json + catalog.json.

Generate Target Artifacts

# On your PR branch
dbt build --target dev
dbt docs generate --target dev

Run Recce

recce server \
  --base-profile dev_base \
  --current-profile dev

# Opens browser at http://localhost:8000

The Recce UI shows your dbt lineage graph. Click any modified model to see checks.

Recce Checks

Value Diff

Compares specific columns between base and target:

Model: orders_by_customer
Column: total_amount
Base: 412 rows, avg=$45.20, total=$18,622
Target: 412 rows, avg=$47.15, total=$19,425
Diff: +4.3% average, +5.2% total

Click into the diff to see which customer IDs have changed values.

Row Count Diff

Model: stg_orders
Base rows: 98,432
Target rows: 97,891
Diff: -541 rows (-0.55%)

Schema Diff

Model: marts.orders
Added columns: order_month (DATE)
Removed columns: none
Type changes: none

Profile Diff

Statistical comparison of column distributions:

Column: amount
                Base        Target      Change
avg             45.20       47.15       +4.3%
stddev          23.10       24.55       +6.3%
min             0.00        0.00        0%
max             1250.00     1250.00     0%
null_rate       0.02%       0.02%       0%

Top-K Diff

For categorical columns, shows distribution changes:

Column: status
Value       Base %      Target %    Change
completed   72.3%       71.1%       -1.2%
pending     18.4%       19.8%       +1.4%
cancelled   9.3%        9.1%        -0.2%

Running Checks via CLI

For CI, run checks non-interactively:

# Row count check
recce run --checks row_count_diff \
  --base-profile dev_base \
  --current-profile dev \
  --select orders_by_customer

# Value diff for specific columns
recce run --checks value_diff \
  --models "marts.*" \
  --columns "revenue,order_count"

# All checks on modified models only
recce run \
  --base-profile dev_base \
  --current-profile dev \
  --select state:modified+  # dbt state selection

Defining Check Files

For repeatable PR validation, define checks in recce.yml:

# recce.yml
checks:
  - name: Revenue should not change by more than 5%
    type: value_diff
    model: marts.daily_revenue
    columns:
      - total_revenue
    threshold: 0.05   # 5% change triggers failure

  - name: Order count diff
    type: row_count_diff
    model: marts.orders_by_customer
    warn_threshold: 0.01    # warn at 1% change
    fail_threshold: 0.05    # fail at 5% change

  - name: Critical columns have no nulls
    type: value_diff
    model: staging.stg_orders
    columns:
      - order_id
      - customer_id
    assert_no_null: true

  - name: Status distribution is stable
    type: top_k_diff
    model: staging.stg_orders
    column: status
    k: 10
    threshold: 0.10   # fail if any status value changes by >10%

Run defined checks:

recce run --config recce.yml \
  --base-profile dev_base \
  --current-profile dev

GitHub Actions CI Integration

# .github/workflows/recce.yml
name: Recce PR Check

on:
  pull_request:
    paths:
      - 'models/**'
      - 'dbt_project.yml'

jobs:
  recce:
    runs-on: ubuntu-latest
    permissions:
      pull-requests: write

    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0  # needed for git diff

      - name: Setup Python
        uses: actions/setup-python@v5
        with:
          python-version: '3.11'

      - name: Install dbt + Recce
        run: |
          pip install dbt-bigquery recce

      - name: Configure GCP credentials
        uses: google-github-actions/auth@v2
        with:
          credentials_json: ${{ secrets.GCP_SA_KEY }}

      - name: Build base branch (main)
        run: |
          git checkout main
          dbt deps
          dbt build --target ci_base --select state:modified+
          dbt docs generate --target ci_base
          # Preserve artifacts
          mkdir -p /tmp/base-artifacts
          cp target/manifest.json /tmp/base-artifacts/
          cp target/catalog.json /tmp/base-artifacts/

      - name: Build PR branch
        run: |
          git checkout ${{ github.head_ref }}
          dbt deps
          dbt build --target ci
          dbt docs generate --target ci

      - name: Run Recce checks
        run: |
          recce run \
            --base-manifest /tmp/base-artifacts/manifest.json \
            --base-catalog /tmp/base-artifacts/catalog.json \
            --current-manifest target/manifest.json \
            --current-catalog target/catalog.json \
            --base-profile ci_base \
            --current-profile ci \
            --config recce.yml \
            --output recce-summary.md

      - name: Comment PR with Recce summary
        uses: actions/github-script@v7
        with:
          script: |
            const fs = require('fs');
            const summary = fs.readFileSync('recce-summary.md', 'utf8');
            github.rest.issues.createComment({
              issue_number: context.issue.number,
              owner: context.repo.owner,
              repo: context.repo.repo,
              body: summary
            });

      - name: Fail if checks failed
        run: |
          if grep -q "FAILED" recce-summary.md; then
            echo "Recce checks failed. Review the PR comment for details."
            exit 1
          fi

The PR comment shows:

## Recce Change Summary

### Modified Models: 2

#### marts.orders_by_customer
- ✅ Row count unchanged (412 rows)
- ✅ Schema unchanged
- ⚠️ total_amount changed by 4.3% (above 0% threshold, below 5% fail threshold)

#### staging.stg_orders
- ✅ Row count unchanged (98,432 rows)
- ✅ total_amount: no change
- ✅ status distribution: stable

**Result: 1 warning, 0 failures**

Recce Cloud

For team-based PR review, Recce Cloud adds:

  • Persistent check results linked to PRs
  • GitHub PR status checks (block merge on failures)
  • Shared review history
  • Approval workflow
# Log in to Recce Cloud
recce cloud login

# Run checks and upload
recce run --cloud \
  --github-pull-request-url ${{ github.event.pull_request.html_url }}

The GitHub status check appears as:

✅ recce/checks — All checks passed
  or
❌ recce/checks — 2 checks failed

Selective Model Testing

Test only the models that changed in the PR:

# Using dbt state selection
recce run \
  --select "state:modified+"  \  # modified models and their downstream
  --base-manifest /tmp/base/manifest.json \
  --current-manifest target/manifest.json

This avoids running expensive row diffs on models you didn't touch.

Interpreting Results

Row count change is expected:

stg_events: -2,341 rows (-0.05%)

This is fine if your PR filters out invalid events. Add a comment in the PR:

"Expected: filtering out events with null session_id (2,341 rows)"

Row count change is unexpected:

orders_by_customer: -15,023 rows (-3.6%)

Investigate: was a JOIN changed from LEFT to INNER? Was a filter added?

Value diff on financial columns:

daily_revenue.total_revenue: +$45,231 (+0.3%)

Always validate financial diffs, even small ones. Check if this is:

  • A data backfill that correctly adds historical revenue
  • A logic change that inflates/deflates revenue

Summary

Recce brings code review discipline to data changes. Before merging any dbt PR, reviewers can see exactly which models changed, how many rows were affected, and whether financial metrics shifted. Static tests tell you data is valid. Recce tells you data is correct. Add the GitHub Actions workflow, define thresholds for your critical business metrics in recce.yml, and require Recce checks to pass before merges — the same way you require unit tests to pass.

Read more

Start now free