> For the complete documentation index, see [llms.txt](https://docs.testery.io/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.testery.io/integrations/ci-cd-integration-guide/azure-devops-pipelines.md).

# Azure DevOps Pipelines

Run Testery from an Azure Pipelines build, publish the results to the pipeline's Tests tab, and show a Testery status check on the pull request.

Azure DevOps works with Testery at three levels. You can use any one of them on its own.

1. **Trigger a test run** from a pipeline with the Testery CLI.
2. **Publish the results** back into the pipeline's **Tests** tab.
3. **Show a Testery status check** on the pull request with the [Azure DevOps integration](/integrations/built-in-integrations-with-testery/azure-devops.md).

## Before you start

1. Add your Testery API token as a secret pipeline variable named `TesteryToken`. Follow the Azure DevOps steps for [secret variables](https://docs.microsoft.com/en-us/azure/devops/pipelines/process/variables?view=azure-devops\&tabs=yaml%2Cbatch#secret-variables). The token is on the **Settings** page under **Integrations**.
2. Make sure the build agent has Python 3 available. Microsoft-hosted agents do.
3. Install Testery CLI **1.20.0 or later** with `pip install --upgrade "testery>=1.20.0"`. Publishing results to the Tests tab relies on the JUnit output added in 1.20.0.

The `PowerShell@2` examples below use forward slashes in paths, so they run unchanged on Windows, Linux and macOS agents. On Linux and macOS the task runs them with PowerShell Core.

## Trigger a test run

Add a step that installs the CLI, uploads the build (for compiled frameworks such as .NET) and creates the test run. Update the project key, environment and paths to match your project.

```yaml
- task: PowerShell@2
  displayName: Run tests in Testery
  inputs:
    targetType: inline
    script: |
      pip -q install --upgrade "testery>=1.20.0"
      testery upload-build-artifacts --token "${env:TESTERY_TOKEN}" --project "example-specflow-dotnet-core" --path "$(Build.SourcesDirectory)/Web.Specs/bin/Release/net8.0/" --build-id "$(Build.BuildNumber)" --zip-dir
      testery create-test-run --token "${env:TESTERY_TOKEN}" --git-ref "$(Build.SourceVersion)" --project "example-specflow-dotnet-core" --environment "dev" --build-id "$(Build.BuildNumber)" --wait-for-results --fail-on-failure
  env:
    TESTERY_TOKEN: $(TesteryToken)
```

{% hint style="info" %}
`--wait-for-results` keeps the step running until Testery finishes, and `--fail-on-failure` fails the pipeline step when any test fails. Leave both off if you only want to kick the run off.
{% endhint %}

Projects whose source is in Azure Repos do not need `upload-build-artifacts` for interpreted frameworks such as Playwright, Cypress or PyTest. Connect the repository through the [Azure DevOps integration](/integrations/built-in-integrations-with-testery/azure-devops.md) and Testery fetches the commit itself.

## Publish results to the Tests tab

`testery report-test-run --output junit` writes the run as JUnit XML, which Azure DevOps can publish as a native test report. The results then show in the pipeline run's **Tests** tab with pass, fail and skip counts, durations and failure messages.

```yaml
- task: PowerShell@2
  displayName: Run tests in Testery
  inputs:
    targetType: inline
    script: |
      pip -q install --upgrade "testery>=1.20.0"
      $gitRef = if ($env:SYSTEM_PULLREQUEST_SOURCECOMMITID) { $env:SYSTEM_PULLREQUEST_SOURCECOMMITID } else { "$(Build.SourceVersion)" }
      $output = testery create-test-run --token "${env:TESTERY_TOKEN}" --git-ref "$gitRef" --project "example-playwright" --environment "dev" --build-id "$(Build.BuildNumber)"
      Write-Host $output
      $testRunId = ($output | Select-String -Pattern 'test_run_id:\s*(\d+)').Matches[0].Groups[1].Value
      testery report-test-run --token "${env:TESTERY_TOKEN}" --test-run-id $testRunId --output junit --outfile "$(Build.ArtifactStagingDirectory)/testery-junit.xml" --fail-on-failure
  env:
    TESTERY_TOKEN: $(TesteryToken)

- task: PublishTestResults@2
  displayName: Publish Testery results
  condition: always()
  inputs:
    testResultsFormat: JUnit
    testResultsFiles: testery-junit.xml
    searchFolder: $(Build.ArtifactStagingDirectory)
    testRunTitle: Testery
    failTaskOnFailedTests: false
```

`report-test-run` waits for the run to finish before it writes the file, so it does not need `--wait-for-results` on `create-test-run`. With `--fail-on-failure` it writes the JUnit file first and then exits non-zero if any test failed, so the step fails but the file is still there to publish. Keep `condition: always()` on the publish task so it still runs after the Testery step fails.

On a pull request build the script tests the pull request's source commit and falls back to `$(Build.SourceVersion)` on other builds. See [Show a Testery status check on the pull request](#show-a-testery-status-check-on-the-pull-request).

{% hint style="warning" %}
`--output junit` needs Testery CLI **1.20.0 or later**. Version 1.19.0 and earlier accept the option but quietly write SonarQube XML, which `PublishTestResults@2` cannot read. Install with `pip install --upgrade "testery>=1.20.0"` to enforce the minimum version.
{% endhint %}

## Show a Testery status check on the pull request

When the project's repository is connected through the [Azure DevOps integration](/integrations/built-in-integrations-with-testery/azure-devops.md), Testery posts a status to the commit it tested and to any active pull request whose source commit or merge commit is that commit. The status shows pending while the run is in progress, then succeeded or failed with the pass and fail counts and a link to the run.

The status is named after the environment key: `Testery (<environment-key>)`, for example `Testery (dev)`. Pass `--test-name` to `create-test-run` to replace `Testery`, which gives `<test-name> (<environment-key>)`. Pass `--status-name` to set the whole name yourself. The environment key is not appended to a `--status-name` value.

Use the pipeline above for pull request validation by triggering it from a build validation branch policy. On a pull request build `$(Build.SourceVersion)` is the merge commit Azure DevOps creates, so pass `$(System.PullRequest.SourceCommitId)` as the git ref to test the pull request's own commit. Fall back to `$(Build.SourceVersion)` because the pull request variable is only set on pull request builds:

```powershell
$gitRef = if ($env:SYSTEM_PULLREQUEST_SOURCECOMMITID) { $env:SYSTEM_PULLREQUEST_SOURCECOMMITID } else { "$(Build.SourceVersion)" }
testery create-test-run --token "${env:TESTERY_TOKEN}" --git-ref "$gitRef" --project "example-playwright" --environment "dev"
```

Testery also matches the merge commit to its pull request, so the status still reaches the pull request if you pass `$(Build.SourceVersion)`, but the run then tests the merged result rather than the branch. Full branch refs such as `refs/heads/main` (the value of `$(Build.SourceBranch)`) are accepted and treated as the branch name.

To block merges until Testery passes, add a **Status check** branch policy on the target branch and pick the Testery status (genre `testery`, name `Testery (<environment-key>)` or your `--test-name` / `--status-name` value) once it has been posted at least once.

## Complete example

A pull request validation pipeline for a Playwright project whose source lives in Azure Repos:

```yaml
trigger:
  - main

pool:
  vmImage: ubuntu-latest

variables:
  TESTERY_PROJECT: example-playwright
  TESTERY_ENVIRONMENT: dev

steps:
  - task: UsePythonVersion@0
    inputs:
      versionSpec: '3.x'

  - script: pip -q install --upgrade "testery>=1.20.0"
    displayName: Install Testery CLI

  - script: |
      set -e
      GIT_REF="${SYSTEM_PULLREQUEST_SOURCECOMMITID:-$BUILD_SOURCEVERSION}"
      OUTPUT=$(testery create-test-run --token "$TESTERY_TOKEN" --git-ref "$GIT_REF" --project "$TESTERY_PROJECT" --environment "$TESTERY_ENVIRONMENT" --build-id "$BUILD_BUILDNUMBER")
      echo "$OUTPUT"
      TEST_RUN_ID=$(echo "$OUTPUT" | sed -n 's/.*test_run_id: *\([0-9]*\).*/\1/p')
      testery report-test-run --token "$TESTERY_TOKEN" --test-run-id "$TEST_RUN_ID" --output junit --outfile "$BUILD_ARTIFACTSTAGINGDIRECTORY/testery-junit.xml" --fail-on-failure
    displayName: Run tests in Testery
    env:
      TESTERY_TOKEN: $(TesteryToken)

  - task: PublishTestResults@2
    displayName: Publish Testery results
    condition: always()
    inputs:
      testResultsFormat: JUnit
      testResultsFiles: testery-junit.xml
      searchFolder: $(Build.ArtifactStagingDirectory)
      testRunTitle: Testery
```