Test Selection

Test Selection allows you to intelligently run only the tests that are relevant to your code changes. By defining rules in a testery.yml file, you can automatically select which tests to run based on which source files have changed.

Overview

When you push code changes to your repository, Testery can analyze which files were modified and automatically determine which tests should run. This helps you:

Save time by running only relevant tests instead of your entire test suite
Get faster feedback on changes that affect specific features
Reduce costs by minimizing unnecessary test execution
Maintain confidence by ensuring related tests always run when code changes

How It Works

Test Selection uses rules defined in a testery.yml configuration file at the root of your repository. Each rule specifies:

When to apply the rule (which source files trigger it)
Then what tests to run (using tags or filters)

Testery compares your commit's changed files against these rules and selectively runs the appropriate tests.

Configuration

Basic Structure

Create a testery.yml file in your repository root with the following structure:

test_selection:
  suites:
    - default:
      rules:
        - ruleName:
          when:
            - src_changes:
                - pattern1
                - pattern2
          then:
            - include_tags:
                - tag1
                - tag2

Example Configuration

Here's a real-world example showing how to configure test selection rules:

test_selection:
  suites:
    - default:
      rules:
        - authenticationRule:
          when:
            - src_changes:
                - src/auth/**/*
                - src/login/**/*
          then:
            - include_tags:
                - authentication
                - login

        - paymentRule:
          when:
            - src_changes:
                - src/payment/**/*
                - src/checkout/**/*
          then:
            - include_tags:
                - payment
                - checkout

        - databaseRule:
          when:
            - src_changes:
                - src/repositories/**/*
                - src/models/**/*
          then:
            - test_filters:
                - Tests.Integration.Database.*

Rule Components

When Conditions

The when section defines which file changes should trigger the rule. Currently supported:

src_changes

Specifies file path patterns to watch for changes. Supports two pattern types:

Glob Patterns (default):

when:
  - src_changes:
      - src/**/*.js           # All JavaScript files in src
      - components/**/*.tsx   # All TypeScript React files
      - config/*.json         # JSON files in config directory

Regex Patterns: Prefix with regex: to use regular expressions:

when:
  - src_changes:
      - regex: .*Grid\.[a-z0-9]+  # Files containing "Grid" with any extension
      - regex: ^src/.*Test\.cs$   # C# test files in src directory

Then Actions

The then section defines which tests to run when the rule matches. You can use:

include_tags

Run tests with specific tags:

then:
  - include_tags:
      - smoke
      - critical

exclude_tags

Exclude tests with specific tags:

then:
  - exclude_tags:
      - slow
      - flaky

test_filters

Run tests matching specific name patterns:

then:
  - test_filters:
      - MyApp.Tests.Unit.*
      - MyApp.Tests.Integration.Auth.*

Multiple Rules

You can define multiple rules, and Testery will combine the results. If your changes match multiple rules, all matching tests will be selected (union behavior).

test_selection:
  suites:
    - default:
      rules:
        - frontendRule:
          when:
            - src_changes:
                - frontend/**/*
          then:
            - include_tags:
                - ui
                - e2e

        - backendRule:
          when:
            - src_changes:
                - backend/**/*
          then:
            - include_tags:
                - api
                - integration

        - sharedRule:
          when:
            - src_changes:
                - shared/**/*
          then:
            - include_tags:
                - ui
                - api

If changes are made to both frontend/ and shared/ files, tests tagged with ui, e2e, and api will all run.

Best Practices

1. Start with Broad Rules

Begin with high-level rules covering major components:

rules:
  - uiRule:
    when:
      - src_changes:
          - src/components/**/*
    then:
      - include_tags:
          - ui

2. Add Specific Rules for Critical Paths

Create targeted rules for high-risk areas:

rules:
  - paymentProcessingRule:
    when:
      - src_changes:
          - src/payment/processor.ts
    then:
      - include_tags:
          - payment
          - critical

3. Use Regex for Complex Patterns

When glob patterns aren't sufficient, use regex:

rules:
  - generatedFilesRule:
    when:
      - src_changes:
          - regex: .*\.generated\.(ts|js)$
    then:
      - include_tags:
          - generated-code

4. Combine Tags and Filters

Use both tags and filters for fine-grained control:

rules:
  - databaseMigrationRule:
    when:
      - src_changes:
          - migrations/**/*
    then:
      - include_tags:
          - database
      - test_filters:
          - Tests.Migrations.*

5. Consider Dependencies

Include tests for code that depends on changed files:

rules:
  - coreLibraryRule:
    when:
      - src_changes:
          - lib/core/**/*
    then:
      - include_tags:
          - core
          - integration  # Run integration tests too

Enabling Test Selection

To enable test selection for a test run:

Add a testery.yml file to your repository with test selection rules
Ensure your tests are tagged appropriately
Enable the "Apply Test Selection Rules" option when creating or configuring your test run in Testery

When test selection is enabled and rules are configured, Testery will:

Detect which files changed in your commit
Evaluate your test selection rules
Identify which rules match the changed files
Run only the tests specified by the matching rules

Common Patterns

Running All Tests for Specific Files

rules:
  - configChangeRule:
    when:
      - src_changes:
          - config/**/*
          - *.config.js
    then:
      - include_tags:
          - all

Framework-Specific Patterns

Pytest:

rules:
  - pytestRule:
    when:
      - src_changes:
          - src/**/*.py
    then:
      - test_filters:
          - tests/unit/test_*.py

NUnit:

rules:
  - nunitRule:
    when:
      - src_changes:
          - src/**/*.cs
    then:
      - test_filters:
          - MyApp.Tests.Unit.*

Cypress:

rules:
  - cypressRule:
    when:
      - src_changes:
          - src/components/**/*
    then:
      - include_tags:
          - '@component-tests'

Troubleshooting

No Tests Running

If no tests run when you expect them to:

Check that your testery.yml is in the repository root
Verify file patterns match your actual file paths
Ensure tags in rules match tags on your tests
Check that "Apply Test Selection Rules" is enabled for your test run

Too Many Tests Running

If more tests run than expected:

Review your patterns for overly broad matches
Use more specific glob patterns or regex
Consider using exclude_tags to filter out tests
Check that multiple rules aren't combining unexpectedly

Pattern Not Matching

If patterns don't seem to match your files:

Verify file paths are relative to repository root
Test glob patterns (they're case-sensitive)
For regex patterns, ensure the regex: prefix is present
Check that special characters are properly escaped