---
title: CI Insights Setup – GitHub Actions
description: Enable Mergify CI Insights and configure flaky test detection using GitHub Actions.
---

import { Image } from "astro:assets"
import JobsScreenshot from "../../../images/ci-insights/jobs.png"

This page explains how to enable CI Insights for a repository using GitHub
Actions and how to configure repeated test executions for flaky test detection.

## Enabling CI Insights for GitHub Actions

1. Enable CI Insights on your repositories by visiting the [GitHub Integration
   page](https://dashboard.mergify.com/integrations/github)
   ([docs](/integrations/github)).

2. In order to upload your test results, you'll need to [configure your
   test](#test-framework-configuration). Before doing so, create an application
   key with `ci` scope as described in the [API Usage documentation](/api/usage)
   and set it as `MERGIFY_TOKEN` in your GitHub Actions secrets.

   This key will be used to upload test reports.

3. Click on `CI Insights` in the Mergify dashboard navigation.
   You should start seeing your GitHub Actions job runs appear:

   <Image src={JobsScreenshot} alt="CI Insights Jobs" />

## Setting Up Flaky Test Detection

To effectively detect flaky tests, you need to run the same tests multiple
times on the same code (identified by the SHA1 of the repository). This section
explains how to set up your CI to systematically detect flaky tests.

### Prerequisites

Before setting up flaky test detection, ensure you have:

1. Enabled [CI Insights](#enabling-ci-insights-for-github-actions) for your repository

2. Created an application key with `ci` scope and set it as `MERGIFY_TOKEN` in GitHub Actions secrets

3. Configured test integration (e.g., `pytest-mergify` or `mergify cli upload`)
   for your test framework

### GitHub Actions Setup

The recommended approach is to use a scheduled workflow that runs your test
suite multiple times on the default branch. Here's an example configuration
that runs tests twice daily with 5 parallel executions:

- Example with native integration (e.g. `pytest-mergify`)

```yaml
name: Continuous Integration
on:
  pull_request:
  schedule:
    - cron: '0 */12 * * 1-5'  # Every 12 hours, Monday to Friday

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v4

      - name: Setup your environment
        # Add your specific setup steps here (Python, Node.js, etc.)

      - name: Run Tests
        env:
          MERGIFY_TOKEN: ${{ secrets.MERGIFY_TOKEN }}
          RUN_COUNT: 5  # Number of times to run each test
        run: |
          # Run your test suite multiple times to detect flakiness
          set +e
          failed=0
          for i in $(seq "$RUN_COUNT"); do
            pytest tests/
            exit_code=$?
            if [ $exit_code -ne 0 ]; then
              failed=1
            fi
          done
          exit $failed
```

- Examples with `mergify cli upload` integration

```yaml
name: CI
on:
  pull_request:
  schedule:
    - cron: '0 */12 * * 1-5'  # Every 12 hours, Monday to Friday

jobs:
  test:
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v4

      - name: Setup your environment
        # Add your specific setup steps here (Python, Node.js, etc.)

      # Modified: Run tests multiple times for flaky detection
      - name: Run tests
        env:
          # Run tests 5 times on schedule, once on pull_request
          RUN_COUNT: ${{ github.event_name == 'schedule' && 5 || 1 }}
        run: |
          # Run the test suite based on RUN_COUNT
          for i in $(seq $RUN_COUNT); do
            echo "Running test suite - attempt $i of $RUN_COUNT"

            # Run your test command with unique output file, e.g.:
            # npm test -- --reporters=default --reporters=jest-junit --outputFile=test-results-$i.xml
          done

      # Upload all test results to CI Insights
      - name: Upload test results
        if: always()
        uses: mergifyio/gha-mergify-ci@v6
        with:
          flaky_test_detection: true
          token: ${{ secrets.MERGIFY_TOKEN }}
          report_path: test-results-*.xml
```

### Configuration Tips

- **Frequency**: Running twice daily (every 12 hours) provides a good balance
  between detection accuracy and resource usage

- **Default Branch Only**: Focus on the default branch where flaky tests have
  the most impact

- **Weekday Schedule**: The example runs Monday to Friday (`1-5`) to avoid
  running when no changes are made on the code

- **Job name**: Ensure the job name is the same as the one running tests on your pull request,
  allowing CI-Insights to identify them. It can be overriden with `MERGIFY_JOB_NAME` if needed.

### Key Environment Variables

- `MERGIFY_TOKEN`: Application key with `ci` scope for uploading test results to CI Insights
- `RUN_COUNT`: Number of times to execute each test within a single job
- `MERGIFY_JOB_NAME`: The job name reported to CI Insights