Set Up Performance Gates in CI/CD with Leapwork Performance

This guide shows how to create threshold-based pass/fail gates for Leapwork Performance in a CI/CD pipeline.

It assumes that your pipeline can already trigger a Leapwork Performance run and poll for completion.

What this guide covers

• How to define performance level thresholds in Leapwork Performance.

• How to use those threshold values as release gates in CI/CD.

• How to fail a pipeline when run results exceed the limits you define.

Before you start

• You need a timeline that already runs successfully in Leapwork Performance.

• You need a Leapwork Performance API key. An admin in your organization can generate this in the Admin Portal under Profile > Integration API Keys.

• Your CI/CD pipeline must already be able to submit a run and poll the terminal result.

• You should agree internally which metrics will block a release, for example errors, average, p95, or p99.

How performance gates work

A performance gate in CI/CD has two parts:

1. You define acceptable response-time bands in Leapwork Performance.

2. Your pipeline evaluates the returned run metrics against those same numeric limits.

Leapwork Performance exposes the run outcome and step-level metrics through the CI/CD integration API. The terminal response includes fields such as:

• status

• errors

• average

• median

• p95

• p99

• max

Use these values to decide whether the pipeline should continue or fail.

Leapwork Performance stores and visualizes threshold bands in the product. In CI/CD, you enforce those thresholds by comparing the returned run metrics against the same numeric values in your pipeline logic.

Step 1: Define threshold bands in Leapwork Performance

1. Open the project that contains your timeline.

2. Open the timeline you want to use as a release gate.

3. Select the sequence on the timeline.

4. In the right-side editor, locate Performance level thresholds.

5. Set the response-time limits for the sequence.

6. Save the timeline.

Each sequence on a timeline has three performance bands:

• Satisfactory: response times from 0 up to the satisfactory upper limit.

• Tolerable: response times from the satisfactory limit up to the tolerable upper limit.

• Frustrating: response times above the tolerable limit.

Default thresholds for a new timeline sequence are:

A practical example:

Step 2: Decide what should fail the pipeline

Leapwork Performance returns several metrics. Not all of them need to be hard gates.

A good starting policy is:

For most teams, the safest first production gate is:

• Fail if the run status is not Finished.

• Fail if any step reports errors > 0.

• Fail if any critical step has p95 above the Tolerable limit.

Step 3: Run the test from CI/CD

Use the CI/CD integration API to:

1. Submit a run.

2. Poll for completion.

3. Evaluate the terminal response.

The terminal poll response includes a metadata array grouped by track item. Each step contains the values you need for gating.

Example terminal fields used for gates:

Step 4: Add a gate evaluation step to your pipeline

The reference integration scripts already stop the pipeline if the run status is not Finished.

To turn the run into a threshold-based release gate, add one more step after polling completes. That step should:

1. Fetch the terminal run response from pollResultUrl.

2. Loop through the returned step metrics.

3. Compare the metrics to your threshold values.

4. Exit with a non-zero code when a rule is violated.

PowerShell example

Use this example in Azure DevOps, Jenkins on Windows, or any PowerShell-based runner.

param(
    [Parameter(Mandatory = $true)][string]$ApiKey,
    [Parameter(Mandatory = $true)][string]$PollResultUrl,
    [int]$SatisfactoryMs = 250,
    [int]$TolerableMs = 1000
)
$headers = @{ Authorization = "X-API-KEY $ApiKey" }
$result = Invoke-RestMethod -Uri $PollResultUrl -Headers $headers -Method Get
if ($result.status -ne 'Finished') {
    throw "Run did not finish successfully. Status: $($result.status)"
}
$failures = New-Object System.Collections.Generic.List[string]
$warnings = New-Object System.Collections.Generic.List[string]
foreach ($track in $result.metadata) {
    foreach ($step in $track.metadata) {
        $name = if ($step.title) { $step.title } else { $step.stepId }
        if (($step.errors | ForEach-Object { [int]$_ }) -gt 0) {
            $failures.Add("$name: errors=$($step.errors)")
        }
        if (($step.p95 | ForEach-Object { [int]$_ }) -gt $TolerableMs) {
            $failures.Add("$name: p95=$($step.p95) ms exceeded tolerable limit $TolerableMs ms")
        }
        if (($step.average | ForEach-Object { [int]$_ }) -gt $SatisfactoryMs) {
            $warnings.Add("$name: average=$($step.average) ms is above satisfactory limit $SatisfactoryMs ms")
        }
    }
}
if ($warnings.Count -gt 0) {
    Write-Host "Warnings:"
    $warnings | ForEach-Object { Write-Host "  $_" }
}
if ($failures.Count -gt 0) {
    Write-Host "Performance gate failed:"
    $failures | ForEach-Object { Write-Host "  $_" }
    exit 1
}
Write-Host "Performance gate passed."

Bash example

Use this example in Jenkins on Linux, GitHub Actions, or any Bash-based runner with jq installed.

#!/usr/bin/env bash
set -euo pipefail
: "${API_KEY:?API_KEY is required}"
: "${POLL_RESULT_URL:?POLL_RESULT_URL is required}"
SATISFACTORY_MS="${SATISFACTORY_MS:-250}"
TOLERABLE_MS="${TOLERABLE_MS:-1000}"
response="$(curl -sS -H "Authorization: X-API-KEY ${API_KEY}" "${POLL_RESULT_URL}")"
status="$(echo "$response" | jq -r '.status')"
if [ "$status" != "Finished" ]; then
  echo "Run did not finish successfully. Status: $status"
  exit 1
fi
warnings="$(echo "$response" | jq -r --argjson satisfactory "$SATISFACTORY_MS" '
  [
    .metadata[]?.metadata[]? as $step
    | ($step.title // $step.stepId) as $name
    | select(($step.average // 0) > $satisfactory)
    | "\($name): average=\($step.average) ms is above satisfactory limit \($satisfactory) ms"
  ] | .[]')"
failures="$(echo "$response" | jq -r --argjson tolerable "$TOLERABLE_MS" '
  [
    .metadata[]?.metadata[]? as $step
    | ($step.title // $step.stepId) as $name
    | if ($step.errors // 0) > 0 then
        "\($name): errors=\($step.errors)"
      elif ($step.p95 // 0) > $tolerable then
        "\($name): p95=\($step.p95) ms exceeded tolerable limit \($tolerable) ms"
      else empty end
  ] | .[]')"
if [ -n "$warnings" ]; then
  echo "Warnings:"
  printf '  %s
' "$warnings"
fi
if [ -n "$failures" ]; then
  echo "Performance gate failed:"
  printf '  %s
' "$failures"
  exit 1
fi
echo "Performance gate passed."

Recommended implementation pattern

For a maintainable setup, store your threshold values as pipeline variables and keep them aligned with the values configured in Leapwork Performance.

Example:

This gives you a predictable gate model:

• Satisfactory helps you detect early drift.

• Tolerable acts as the release gate.

• Frustrating remains visible in the product for analysis and reporting.

Example release policy

A typical release policy for a customer-facing API might be:

• Pass only if the run finishes successfully.

• Pass only if all steps have errors = 0.

• Fail if any critical step has p95 > 1500 ms.

• Warn if average response time for any critical step is above 500 ms.

This gives teams a clear, automated release signal without overreacting to a single outlier.

Troubleshooting

• If the pipeline fails before gate evaluation, confirm that the run reached a terminal Finished status.

• If the gate fails unexpectedly, inspect the returned metadata fields for the affected step and compare them with your configured threshold values.

• If the pipeline uses different threshold values from the timeline, update the pipeline variables so they match the values defined in Leapwork Performance.

• If you want to gate only specific APIs, filter the evaluation script to a list of critical step names instead of checking every step.

Next step

If you have not yet connected Leapwork Performance to your pipeline, start with the CI/CD integration reference. After that is in place, add a gate evaluation step using the examples above