Codecov

Codecov vs Coveralls: Coverage Reporting for GitHub Projects

HelpMeTest

HelpMeTest

— 4 min read
Codecov vs Coveralls: Coverage Reporting for GitHub Projects

Codecov and Coveralls both receive coverage reports from CI, display trends over time, and post PR comments showing coverage changes. Codecov offers more powerful PR analytics and supports more coverage formats. Coveralls is simpler and free for open source. This guide covers setting up both and integrating with GitHub.

Running coverage tools locally is useful. Tracking coverage over time, seeing how PRs affect coverage, and blocking merges when coverage drops—that requires a coverage service. Codecov and Coveralls are the two most popular options for GitHub-hosted projects.

What Coverage Services Provide

Beyond the local coverage report, both services add:

  • Historical tracking: Coverage percentage over time and across branches
  • PR comments: Automatic comments showing coverage delta (increased/decreased)
  • GitHub status checks: A "coverage/codecov" check that can block merges
  • Badge generation: [![codecov](...)](#) in README
  • File-level drill-down: Which files lost coverage in this PR
  • Team dashboards: Aggregate coverage across multiple repositories

Codecov

Codecov is the more feature-rich option. It supports coverage reports from any tool that generates XML, LCOV, or Cobertura format.

Setup: GitHub Actions

# .github/workflows/test.yml
name: Tests

on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Run tests with coverage
        run: |
          # Python example
          pytest --cov=myapp --cov-report=xml tests/

          # JavaScript example (Jest)
          # npm test -- --coverage --coverageReporters=lcov

          # Java (Maven)
          # mvn verify

      - name: Upload to Codecov
        uses: codecov/codecov-action@v4
        with:
          token: ${{ secrets.CODECOV_TOKEN }}
          file: coverage.xml     # Or lcov.info, jacoco.xml
          flags: unittests       # Tag this upload
          name: codecov-umbrella
          fail_ci_if_error: false  # Don't fail build on upload error

Codecov Configuration

Add codecov.yml to the repository root:

# codecov.yml
coverage:
  status:
    project:
      default:
        target: 80%           # Overall project coverage target
        threshold: 2%         # Allow 2% drop before failing
    patch:
      default:
        target: 70%           # New code in PR must be 70% covered
        threshold: 5%

comment:
  layout: "reach,diff,files,tree"  # What to show in PR comment
  behavior: default
  require_changes: false     # Comment on every PR, not just ones with changes

ignore:
  - "**/*.test.js"
  - "**/migrations/**"
  - "**/fixtures/**"
  - "docs/**"

Patch Coverage

Codecov's "patch coverage" is its killer feature: it calculates coverage only for the lines changed in the PR, not the entire codebase.

A PR that adds 50 lines of new code with only 30 lines tested = 60% patch coverage. This directly answers "did this PR's author test their new code?" without being affected by pre-existing coverage debt.

Configure patch coverage separately from project coverage:

coverage:
  status:
    patch:
      default:
        target: 80%   # New code must be 80% covered

PR Comment Example

Codecov posts comments like:

Coverage Report - 85.23% (+0.42%)

  Files      Coverage
+ src/api/orders.py   92% (+5%)
+ src/models/user.py  88% (+2%)
- src/utils/email.py  61% (-8%)

Patch Coverage: 78.5% of added lines covered

Lines in red (decreasing coverage) highlight exactly where test gaps are in the PR.

Codecov GitHub App

Install the Codecov GitHub App for automatic PR status checks. After installing:

  1. The coverage/codecov/patch and coverage/codecov/project checks appear on every PR
  2. Configure branch protection to require these checks before merging
  3. Coverage drops block merges automatically—no manual review required

Coveralls

Coveralls is simpler and free for open-source repositories. It's a good choice for small projects that don't need Codecov's advanced analytics.

Setup: GitHub Actions

name: Tests

on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: '3.11'

      - name: Install dependencies
        run: pip install pytest pytest-cov coveralls

      - name: Run tests
        run: pytest --cov=myapp --cov-report=lcov tests/

      - name: Upload to Coveralls
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
        run: coveralls --service=github

For JavaScript projects:

- name: Run Jest with coverage
  run: npm test -- --coverage --coverageReporters=lcov

- name: Upload to Coveralls
  uses: coverallsapp/github-action@v2
  with:
    github-token: ${{ secrets.GITHUB_TOKEN }}
    path-to-lcov: coverage/lcov.info

Coveralls Configuration

# .coveralls.yml
service_name: github-actions
repo_token: ${{ COVERALLS_REPO_TOKEN }}  # Only needed for private repos

Coveralls is simpler to configure than Codecov—it reads LCOV format and posts a basic summary. No separate service token needed for public repos on GitHub Actions (uses GITHUB_TOKEN).

Language-Specific Coverage Upload Examples

JavaScript/TypeScript (Jest)

- name: Test with Jest
  run: npx jest --coverage --coverageReporters=lcov,json-summary

- name: Upload to Codecov
  uses: codecov/codecov-action@v4
  with:
    token: ${{ secrets.CODECOV_TOKEN }}
    directory: coverage/   # Jest writes to ./coverage by default

Java (JaCoCo + Maven)

- name: Maven verify
  run: mvn verify

- name: Upload JaCoCo report
  uses: codecov/codecov-action@v4
  with:
    token: ${{ secrets.CODECOV_TOKEN }}
    file: target/site/jacoco/jacoco.xml

Go

- name: Test with coverage
  run: go test ./... -coverprofile=coverage.out -covermode=atomic

- name: Convert to lcov
  run: go tool cover -html=coverage.out -o coverage.html

- name: Upload to Codecov
  uses: codecov/codecov-action@v4
  with:
    token: ${{ secrets.CODECOV_TOKEN }}
    file: coverage.out

Ruby (SimpleCov)

# spec/spec_helper.rb or test/test_helper.rb
require 'simplecov'
require 'simplecov-lcov'

SimpleCov::Formatter::LcovFormatter.config.report_with_single_file = true
SimpleCov.formatter = SimpleCov::Formatter::LcovFormatter
SimpleCov.start 'rails'
- name: Upload to Coveralls
  uses: coverallsapp/github-action@v2
  with:
    github-token: ${{ secrets.GITHUB_TOKEN }}
    path-to-lcov: coverage/lcov.info

Comparing Codecov vs Coveralls

Feature Codecov Coveralls
Open source (free) ✓ ✓
Private repos (free) Limited Limited
Patch coverage ✓ ✗
Sunburst coverage viz ✓ ✗
Carryforward flags ✓ ✗
Setup complexity Medium Low
Coverage formats Most LCOV, simplecov
GitHub App ✓ ✓

Choose Codecov if: You care about patch coverage analysis, want file-level coverage trends, or have complex multi-language setups.

Choose Coveralls if: You want the simplest possible setup for a single-language open-source project.

Carryforward Flags (Codecov)

For projects with separate CI jobs (unit tests, integration tests), Codecov's carryforward flags prevent a coverage drop from showing when only one job runs:

# codecov.yml
flag_management:
  default_rules:
    carryforward: true

coverage:
  status:
    project:
      unit:
        flags:
          - unit
        target: 80%
      integration:
        flags:
          - integration
        target: 70%
# CI: tag uploads with flags
- name: Upload unit test coverage
  uses: codecov/codecov-action@v4
  with:
    token: ${{ secrets.CODECOV_TOKEN }}
    file: unit-coverage.xml
    flags: unit

- name: Upload integration test coverage
  uses: codecov/codecov-action@v4
  with:
    token: ${{ secrets.CODECOV_TOKEN }}
    file: integration-coverage.xml
    flags: integration

Coverage Badges

Both services provide badge URLs for README:

<!-- Codecov -->
[![codecov](https://codecov.io/gh/owner/repo/branch/main/graph/badge.svg?token=TOKEN)](https://codecov.io/gh/owner/repo)

<!-- Coveralls -->
[![Coverage Status](https://coveralls.io/repos/github/owner/repo/badge.svg?branch=main)](https://coveralls.io/github/owner/repo?branch=main)

Summary

Codecov and Coveralls solve the same problem—visibility into coverage trends and PR impact—with different complexity tradeoffs. Codecov's patch coverage analysis is the most actionable metric for code review: it tells you specifically whether the new code in a PR is tested, independent of historical coverage debt. Set it up with branch protection rules to automatically require coverage for new code. Coveralls requires less configuration and works well for simpler setups. Both integrate with any coverage tool that outputs standard formats (LCOV, Cobertura, JaCoCo XML).

Read more

Start now free