Integrations
The GitHub pull request check
A change to the cart template passes code review, the tests are green, and a week later purchase is missing from the report. The pull request check moves that moment before the merge: we replay every pull request on its preview environment with the same journeys you monitor in production, and post the result to GitHub as a check named AnalyticsProof.
What you need
- A project on the Enterprise plan and a website or HbbTV test with at least one recorded journey. The "Connect GitHub" button is on the test page. Editors and above can change the GitHub connection; everyone else sees the button greyed out.
- A preview environment deployed to its own https address for each pull request. It can sit behind a password or an IP allowlist; both are set in the third step.
- Journeys recorded in the time zone of the scan country. If one does not match, the dialog names it and asks you to re-record it before turning the check on.
The setup has three steps: Repository, Preview URL and Access.
Step 1: Repository
Click "Authorize GitHub". A new tab opens where you grant access on GitHub, after which we can see the repositories you are allowed to change. If the AnalyticsProof App is not installed in your organization yet, a "Connect the AnalyticsProof App on GitHub" link appears. When installing, choose "Only select repositories" and tick just the website's repository. Back in the dialog, use "Refresh repositories" and pick it under "Website repository".
Step 2: Preview URL
"Where the preview URL comes from" offers two options.
From GitHub Deployments. Use it when your hosting or pipeline reports pull request deployments to GitHub. We wait for a deployment of the pull request's commit with the state success and an environment URL (environment_url). Until it arrives, the check says it is waiting for the address. If nothing arrives within 10 minutes of the check starting, the check ends as neutral.
From a URL template. Use it when the preview address is predictable. The template takes these placeholders:
| Placeholder | Replaced with |
|---|---|
{branch} |
The branch name in lower case, each run of other characters replaced by a hyphen |
{pr} |
The pull request number |
{sha} |
The full commit hash |
{sha7} |
The first seven characters of the commit hash |
Below the field you see straight away what the address would be for the branch feature/checkout. The template must lead to a public domain over https://, without a port number, without a username or password in the address, and without an IP address. The dialog rejects local names (.local, .internal, .test, .example and similar). A template without any placeholder is accepted, but the dialog warns that every pull request would be tested against the same address.
Step 3: Access
Preview password. If the preview is behind Basic auth, tick "The preview requires a username and password (Basic auth)" and fill both in. The browser sends them only to the preview's own origin, never to other services on the page. The password is stored as plain text and editors can read it in the dialog, so use a separate login that exists only for the preview, not a personal account.
IP allowlist. The "Allow our IP" field holds the address we scan from. It follows the country chosen when the test was created, and pull request scans use the same address. Add it to the preview environment's allowlist.
"Check and save" first tries to open the preview: DNS, TLS handshake, HTTP response, page render and DataLayer. It saves only when that check passes. When it fails, you see which step failed and what to do about it.
There is no pull request at that moment, so the check uses the repository's default branch and its latest commit. The template gets the default branch name, and {pr} becomes zero. A template built only on the pull request number therefore passes only if that address exists. With GitHub Deployments, the latest commit on the default branch needs a successful deployment. If it has none, the dialog offers "Save anyway".
Once saved, the dialog offers "Run check". It replays the journeys against the same address the check used, and "Open the results" leads to the result in the dashboard. It is the quickest way to prove the whole setup before a colleague meets it on their pull request.
What the check reports
The check runs when a pull request is opened, pushed to, reopened or marked ready for review. A draft pull request gets a neutral check and is replayed once it is ready for review. A result for an older commit is not posted if a newer push has arrived in the meantime.
- Every active journey of the test is replayed, each starting at the preview URL. It runs as a dry run: it produces differences and a result for the check, but writes nothing to the journey or its expected state. The API reference has more on dry runs.
- Only what the website pushes directly to
window.dataLayercounts. Requests that tools such as GA4 send on from there stay in the dashboard; a developer changing the website does not own them directly. - Only what the pull request introduced is judged. Differences are compared with the journey's latest production scan, and the pull request comment splits them into "New in this PR" and "Already on production". If production has not been scanned yet, everything counts as new.
- Only critical changes block. Typically that is an event the expected state requires and the step no longer sends (unless you marked it optional). A new event, a missing parameter or a different parameter type are warnings: the comment lists them, but the check stays green. Parameter values are not compared, only their presence and JavaScript type.
- A journey that could not be measured is not green. If a journey's scan fails or measures nothing, the check turns red even without a single change. A journey that does not reach its last step is only a warning: a larger website change probably broke it, not this pull request.
The check and the comment are in English regardless of your dashboard language. The repository's reviewers read them, and the check name has to stay stable for branch protection.
Making the check required
Once the check has passed for the first time, you can make it required on GitHub. In the branch protection rules or a repository ruleset, require status checks and select AnalyticsProof. GitHub only offers checks that have run in the repository recently, which is why this works only after the first run.
Keep in mind that GitHub treats a neutral result as satisfied. A pull request for which no preview was found, a draft, or one where no journey could be compared therefore does not block the merge.
When the check does not do what you expect
| Check title | What it means | What to do |
|---|---|---|
AnalyticsProof β critical dataLayer change |
A journey found a critical change, or could not be measured. | The check's table shows each journey's status; the detail is behind the result link. |
AnalyticsProof β waiting for a preview URL |
No deployment with an address has arrived yet, or the template address could not be used. | Make sure the hosting reports a deployment for the pull request's commit, or fix the template. |
AnalyticsProof β nothing measurable |
No journey could be compared, typically because none has an expected state yet. As soon as at least one journey can be compared, the check is green or red by it, and the table lists the others with the reason. | Wait until the journeys have built their expected state; its state is shown on the journey in the dashboard. |
AnalyticsProof β nothing to run |
The test has no active journey. | Record a journey or resume a paused one. |
AnalyticsProof β could not run |
An error on our side. | The check retries by itself. If it stays, tell us and we will fix it. |
AnalyticsProof β Enterprise only |
The project is not on the Enterprise plan. | The pull request check is part of the Enterprise plan. |
AnalyticsProof β not configured |
The App is installed on the repository, but the repository is not connected to any test. | Connect it in the test's GitHub dialog, or remove the repository from the App's installation. |
AnalyticsProof β will run once the PR is ready for review |
The pull request is a draft. | Mark it ready for review; the check then runs by itself. |
A common cause of a red check is not the dataLayer at all. The preview turns our browser away because the IP is missing from the allowlist or the password is wrong. Run "Check and save" again; it shows exactly which step of the access failed.
Not on GitHub, or want to build your own gate? The API does the same: the guide CI/CD with the API walks through it, and the CI recipes are ready to copy.