Go

Go Integration Testing: Patterns for Real Databases and HTTP APIs

HelpMeTest

HelpMeTest

— 4 min read
Go Integration Testing: Patterns for Real Databases and HTTP APIs

Unit tests verify logic in isolation. Integration tests verify that components work together: your code + database, your service + external API, your handler + middleware stack. Go's testing package supports both, and the patterns for integration testing are as idiomatic as those for unit testing.

Separating Integration Tests

Mark slow integration tests with a build tag or short-mode check to keep them out of the regular go test cycle:

Build tag approach (recommended):

//go:build integration

package db_test

import (
    "testing"
    "database/sql"
    _ "github.com/lib/pq"
)

Run only integration tests:

go test -tags integration ./...

Short mode approach:

func TestDatabaseIntegration(t *testing.T) {
    if testing.Short() {
        t.Skip("skipping integration test")
    }
    // ...
}

Run excluding slow tests:

go test -short ./...

Testing with a Real Database

TestMain for Package Setup

//go:build integration

package db_test

import (
    "database/sql"
    "os"
    "testing"
    _ "github.com/lib/pq"
)

var testDB *sql.DB

func TestMain(m *testing.M) {
    dsn := os.Getenv("TEST_DATABASE_URL")
    if dsn == "" {
        dsn = "postgres://user:pass@localhost:5432/testdb?sslmode=disable"
    }

    var err error
    testDB, err = sql.Open("postgres", dsn)
    if err != nil {
        panic("failed to connect to test database: " + err.Error())
    }
    defer testDB.Close()

    if err = testDB.Ping(); err != nil {
        panic("test database not reachable: " + err.Error())
    }

    // Run migrations
    runMigrations(testDB)

    // Run all tests
    code := m.Run()
    os.Exit(code)
}

Per-Test Isolation with Transactions

Wrap each test in a transaction and roll back at the end — the fastest way to isolate tests:

func newTestTx(t *testing.T) *sql.Tx {
    t.Helper()
    tx, err := testDB.Begin()
    if err != nil {
        t.Fatalf("begin transaction: %v", err)
    }
    t.Cleanup(func() {
        tx.Rollback()
    })
    return tx
}

func TestUserRepository_Create(t *testing.T) {
    tx := newTestTx(t)
    repo := NewUserRepository(tx)

    user := &User{Name: "Alice", Email: "alice@example.com"}
    err := repo.Create(user)
    if err != nil {
        t.Fatalf("Create: %v", err)
    }
    if user.ID == 0 {
        t.Error("expected ID to be set after create")
    }

    found, err := repo.FindByID(user.ID)
    if err != nil {
        t.Fatalf("FindByID: %v", err)
    }
    if found.Name != "Alice" {
        t.Errorf("got name %q, want Alice", found.Name)
    }
    // Rollback happens in Cleanup — no permanent records
}

This approach is fast (no truncation) and safe (each test is isolated).

Per-Test Database Truncation

For tests that can't run in a transaction (e.g., multi-connection tests):

func truncateAll(t *testing.T, db *sql.DB) {
    t.Helper()
    tables := []string{"orders", "users", "products"}
    for _, table := range tables {
        _, err := db.Exec("TRUNCATE TABLE " + table + " CASCADE")
        if err != nil {
            t.Fatalf("truncate %s: %v", table, err)
        }
    }
}

func TestOrderService(t *testing.T) {
    truncateAll(t, testDB)
    // test code
}

testcontainers-go

testcontainers-go spins up real Docker containers for testing — no pre-running database required:

go get github.com/testcontainers/testcontainers-go
func TestWithPostgres(t *testing.T) {
    ctx := context.Background()

    container, err := postgres.Run(ctx,
        "postgres:16",
        postgres.WithDatabase("testdb"),
        postgres.WithUsername("user"),
        postgres.WithPassword("password"),
        testcontainers.WithWaitStrategy(
            wait.ForLog("database system is ready").WithStartupTimeout(60*time.Second),
        ),
    )
    require.NoError(t, err)
    t.Cleanup(func() { container.Terminate(ctx) })

    dsn, err := container.ConnectionString(ctx, "sslmode=disable")
    require.NoError(t, err)

    db, err := sql.Open("postgres", dsn)
    require.NoError(t, err)
    defer db.Close()

    // Run your tests against this fresh, isolated database
    runMigrations(db)
    repo := NewUserRepository(db)
    // ...
}

testcontainers is slower to start than an already-running database but gives you a completely fresh, isolated environment per test run. It's excellent for CI where you want no shared state.

HTTP Handler Integration Tests

Test your HTTP handlers against a real net/http server using httptest:

func TestUserHandler_GetUser(t *testing.T) {
    // Setup
    db := setupTestDB(t)
    repo := NewUserRepository(db)
    handler := NewUserHandler(repo)

    server := httptest.NewServer(handler)
    defer server.Close()

    // Create test data
    user, _ := repo.Create(&User{Name: "Alice", Email: "alice@example.com"})

    // Make real HTTP request
    resp, err := http.Get(server.URL + "/users/" + strconv.Itoa(user.ID))
    if err != nil {
        t.Fatalf("request failed: %v", err)
    }
    defer resp.Body.Close()

    // Assert response
    if resp.StatusCode != http.StatusOK {
        t.Errorf("got status %d, want 200", resp.StatusCode)
    }

    var got User
    json.NewDecoder(resp.Body).Decode(&got)
    if got.Name != "Alice" {
        t.Errorf("got name %q, want Alice", got.Name)
    }
}

ResponseRecorder for Handler-Level Tests

For testing handlers without a network:

func TestUserHandler_Create(t *testing.T) {
    repo := NewFakeUserRepository()  // or a mock
    handler := NewUserHandler(repo)

    body := strings.NewReader(`{"name":"Alice","email":"alice@example.com"}`)
    req := httptest.NewRequest(http.MethodPost, "/users", body)
    req.Header.Set("Content-Type", "application/json")
    w := httptest.NewRecorder()

    handler.ServeHTTP(w, req)

    resp := w.Result()
    if resp.StatusCode != http.StatusCreated {
        t.Errorf("got %d, want 201", resp.StatusCode)
    }
}

httptest.NewRecorder() captures the response without a network connection. Faster than httptest.NewServer but tests the handler in isolation from middleware.

Testing gRPC Services

For gRPC, use bufconn to avoid real network connections:

func TestUserService_gRPC(t *testing.T) {
    buf := bufconn.Listen(1024 * 1024)
    s := grpc.NewServer()
    RegisterUserServiceServer(s, NewUserServiceServer(setupTestDB(t)))

    go func() {
        if err := s.Serve(buf); err != nil {
            t.Errorf("server error: %v", err)
        }
    }()
    t.Cleanup(s.GracefulStop)

    conn, err := grpc.NewClient("passthrough://bufnet",
        grpc.WithContextDialer(func(ctx context.Context, _ string) (net.Conn, error) {
            return buf.DialContext(ctx)
        }),
        grpc.WithTransportCredentials(insecure.NewCredentials()),
    )
    require.NoError(t, err)
    defer conn.Close()

    client := NewUserServiceClient(conn)
    resp, err := client.GetUser(context.Background(), &GetUserRequest{Id: 1})
    require.NoError(t, err)
    assert.Equal(t, "Alice", resp.User.Name)
}

Structuring Integration Tests

A common layout:

├── internal/
│   └── user/
│       ├── repository.go
│       ├── repository_test.go          # unit tests (mocked db)
│       └── repository_integration_test.go  # integration tests (real db)
├── api/
│   └── handler_test.go                # httptest-based integration tests
└── test/
    ├── integration/                   # cross-service integration tests
    │   └── order_flow_test.go
    └── helpers/
        └── db.go                      # shared test DB helpers

Shared Test Helpers

// test/helpers/db.go
//go:build integration

package helpers

import (
    "testing"
    "database/sql"
)

func NewTestDB(t *testing.T) *sql.DB {
    t.Helper()
    db, err := sql.Open("postgres", os.Getenv("TEST_DATABASE_URL"))
    if err != nil {
        t.Fatalf("open db: %v", err)
    }
    t.Cleanup(func() { db.Close() })
    runMigrations(db)
    return db
}

CI Configuration

# .github/workflows/test.yml
jobs:
  unit-tests:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-go@v5
      - run: go test -short ./...

  integration-tests:
    runs-on: ubuntu-latest
    services:
      postgres:
        image: postgres:16
        env:
          POSTGRES_DB: testdb
          POSTGRES_USER: user
          POSTGRES_PASSWORD: password
        options: >-
          --health-cmd pg_isready
          --health-interval 10s
          --health-timeout 5s
          --health-retries 5
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-go@v5
      - env:
          TEST_DATABASE_URL: postgres://user:password@localhost/testdb?sslmode=disable
        run: go test -tags integration ./...

This runs unit tests fast on every push and integration tests separately with a real Postgres service. Keep integration tests in their own job so unit test failures don't block running integration tests.

Read more