CI/CD with the API

A deploy that breaks measurement is best caught before it reaches production. The API can replay your saved journeys against the preview of a specific change, compare the result with the expected state and return a clear pass or fail for the CI job to exit on. This guide shows how to build that flow and where it usually gets stuck.

The flow has four steps: trigger, wait, evaluate the gate, and optionally emit SARIF or a comment.

GitHub App, action or your own calls

On GitHub you have three options, all built on the same API:

• The GitHub App (pull request check) is set up in the dashboard; nothing goes into the repository. Its rule is fixed: it reports only direct dataLayer changes and blocks only critical findings.
• The composite action analyticsproof runs in your workflow. You copy it into your repository as .github/actions/analyticsproof and call it with a uses: line; the workflow to go with it is in the CI recipes. It replays every journey your plan allows to run (paused ones excluded) against the preview, applies the same fixed rule, writes a SARIF file and comments on the pull request.
• Your own API calls work in any CI, and you choose the gate rule. You also write the waiting and the evaluation yourself.

Happy with the default rule on GitHub? Use the App. Want the check inside your workflow, or results in code scanning? The action. A different CI or a different rule means the API.

The key

An editor or higher role creates the key under Project Settings › API Keys. CI needs two scopes: “Trigger scans” and “Read scans & results”. Nothing else. Expiry is optional, and the full key is shown only once, right after creation.

The key starts with ap_live_ and goes in an Authorization: Bearer … or X-API-Key: … header. Store it as a masked variable or secret, never in the repository. The API is part of the Enterprise plan; on a project without it, even a valid key gets 403 tier_forbidden. Details are in the API reference.

Triggering a run against a preview

Trigger a scan with POST /api/v1/targets/{id}/scans; find the test's id via GET /api/v1/targets. When the body carries a url with the preview address, the scan runs as a dry run: it produces diffs and a result for the gate, but nothing is written back to the journey. The expected state, the watched tools and the regression history stay as they were. Without url, the same call starts an ordinary scan of the saved test.

On a website test, a url alone is not enough for a journey. Without a scan type the request runs the test's default scan, a compliance audit of the preview. To replay one journey, add "scanType": "web_datalayer" and optionally a journeyId; without it the test's first runnable journey is used. The response then carries one scanId and its pollUrl.

With "journeys": "all", every journey of the test that your plan allows to run is replayed, and maxJourneys caps how many. The response has a different shape: there is no top-level scanId, only a scans array with a scanId and pollUrl per journey, and a failed array with journeys that could not be started. Your script has to wait for each scans[].pollUrl, as the composite action does. A non-empty failed should fail the job.

Two things worth setting from the start:

• An Idempotency-Key header. A retry with the same key does not start a second scan; it returns the first one. The key identifies one specific request: the test, the journey selection and the preview URL. Keep it unchanged across retries of that request, and do not derive it from the commit hash alone. The same commit can be deployed to a second preview or checked again with different journeys, and with the commit alone as the key you would get back the old scan.
• The scan country. The preview is scanned from the country chosen when the test was created. If the preview sits behind an IP allowlist, allow that country's address.

See dry run in the reference.

Waiting for the result

Read the scan's state with GET /api/v1/scans/{id}?wait=N. The server holds the request open until the scan has finished and its findings are stored, for at most N seconds. The cap is 300 seconds. Waiting around 120 seconds and repeating is more practical, because proxies and runners like to cut long connections.

The scan has finished when its status is one of these:

Repeat until status is final and settled is true. settled: false means your wait ran out while the findings were still being stored. Just ask again.

One key may hold at most five waiting requests at a time; the sixth gets 429 too_many_long_polls. If you run several journeys at once, read them one after another, not in parallel.

Designing the gate

Pass the gate as ?gate= with URL-encoded JSON when reading the scan. Predicates combine with OR: the gate fails if any of them matches. All predicates are in the reference. For a preview and the dataLayer, this is a sensible start:

{ "failOnSeverity": "critical", "failOnBrokenStep": true }

Rules you will not read off the predicate table:

• A scan that ends failed, cancelled or no_data never passes a gate, whatever you set. A scan without data does not get a false “score below threshold” either; it has its own code.
• By default a broken step does not fail the gate. It ends as a warning, and that scan's diffs are not evaluated. If a change breaks the basket button, a gate without failOnBrokenStep passes. That is why we recommend turning it on.
• An event shifted by one step is within tolerance and never fails the gate.
• While a journey has no expected state yet, the dataLayer predicates have nothing to compare against. They cannot fail, and warnings carries datalayer_gate_no_baseline. It pays to act on this warning in CI.
• Without ?gate=, the response contains no gate at all. Your script must therefore treat a missing gate.passed as a failure, not a pass.
• minComplianceScore judges the compliance score, so it belongs with a compliance test. A journey is judged by its diffs.

The scope=direct-datalayer parameter narrows the diffs to what the site pushes directly into window.dataLayer. Requests that GA4 and other tools build from it stay in the dashboard but do not affect the gate. The App and the action use the same narrowing. The comparison covers the presence of events and parameters and their JavaScript type, not their values. The exception is consent signals, where a changed value is a finding of its own.

SARIF and the comment

With ?format=sarif, a finished scan returns a SARIF 2.1.0 document. A scan that has not finished returns the ordinary JSON, so fetch SARIF only after a JSON answer whose status is final and whose settled is true. While findings are still being stored, SARIF could be incomplete. SARIF contains the same findings the gate judges, plus the events shifted by one step that the gate tolerates (critical as error, warning as warning, info as note). It does not carry the verdict; gate.passed still decides the job. Add the same scope so the file matches the gate.

For a pull request or merge request comment, call POST /api/v1/scans/{id}/explain. It returns a plain-language explanation of the diffs without parameter values. Before the scan finishes it answers 409 scan_not_terminal.

Example: GitLab CI and other CIs

The steps are the same in any CI that has curl and jq:

1. Store the key and the test id as masked CI variables.
2. Place the job after the preview deploy, so it knows the preview address.
3. Trigger a dry run with url, "journeys": "all" and an Idempotency-Key built from the test ID, the journey selection and the preview URL. Keep every pollUrl from scans, and fail on a non-empty failed.
4. Read each scan with wait=120, the gate and the scope until it is final and settled. Bound the whole wait with the job timeout.
5. Fail the job when gate.passed is not true for any journey. Print gate.failures and warnings to the log.
6. If your CI does not display SARIF, keep it as a job artifact.

A complete GitLab script is in the CI recipes. It triggers an ordinary scan of the saved test and reads a single scanId. For a preview of one journey, send the body {"url": "…", "scanType": "web_datalayer"}, add the preview URL to the Idempotency-Key and swap minComplianceScore for the journey gate above; the rest of the script stays as it is. For "journeys": "all" it needs a loop over scans[], because the response has no scanId.

Errors and what to do

Codes in gate.failures are not API errors; they are the reasons the gate did not pass. Every code is described in the error overview.