From fe8310509c688bbaaba5de7f533e2c0607eb7fa9 Mon Sep 17 00:00:00 2001 From: Geng Yan Date: Mon, 6 Jan 2025 10:57:28 +0800 Subject: [PATCH] refactor: update README to reflect new features and usage of gylove1994/paths-filter@v2 - Forked from dorny/paths-filter, this version introduces significant changes to the README. - Updated usage instructions to recommend `gylove1994/paths-filter@v2`. - Added new features including support for ignore patterns and complex matching for client and server files. - Enhanced examples to demonstrate the new filtering capabilities and configurations. - Improved clarity and organization of the documentation for better user guidance. --- README.md | 586 ++++-------------------------------------------------- 1 file changed, 40 insertions(+), 546 deletions(-) diff --git a/README.md b/README.md index b5e0f4cf..15935358 100644 --- a/README.md +++ b/README.md @@ -1,567 +1,61 @@ # Paths Changes Filter -[GitHub Action](https://github.com/features/actions) that enables conditional execution of workflow steps and jobs, based on the files modified by pull request, on a feature -branch, or by the recently pushed commits. +This is a fork of [dorny/paths-filter](https://github.com/dorny/paths-filter) -Run slow tasks like integration tests or deployments only for changed components. It saves time and resources, especially in monorepo setups. -GitHub workflows built-in [path filters](https://docs.github.com/en/actions/reference/workflow-syntax-for-github-actions#onpushpull_requestpaths) -don't allow this because they don't work on a level of individual jobs or steps. +## !!!!!!!!!!!!Important!!!!!!!!!!!! -**Real world usage examples:** +Please use `gylove1994/paths-filter@v2`. -- [sentry.io](https://sentry.io/) - [backend.yml](https://github.com/getsentry/sentry/blob/2ebe01feab863d89aa7564e6d243b6d80c230ddc/.github/workflows/backend.yml#L36) -- [GoogleChrome/web.dev](https://web.dev/) - [lint-workflow.yml](https://github.com/GoogleChrome/web.dev/blob/3a57b721e7df6fc52172f676ca68d16153bda6a3/.github/workflows/lint-workflow.yml#L26) -- [blog post Configuring python linting to be part of CI/CD using GitHub actions](https://dev.to/freshbooks/configuring-python-linting-to-be-part-of-cicd-using-github-actions-1731#what-files-does-it-run-against) - [py_linter.yml](https://github.com/iamtodor/demo-github-actions-python-linter-configuration/blob/main/.github/workflows/py_linter.yml#L31) +## What's Changed -## Supported workflows +We add some new features to the original project, such as: -- **Pull requests:** - - Workflow triggered by **[pull_request](https://docs.github.com/en/actions/reference/events-that-trigger-workflows#pull_request)** - or **[pull_request_target](https://docs.github.com/en/actions/reference/events-that-trigger-workflows#pull_request_target)** event - - Changes are detected against the pull request base branch - - Uses GitHub REST API to fetch a list of modified files - - Requires [pull-requests: read](https://docs.github.com/en/actions/using-jobs/assigning-permissions-to-jobs) permission -- **Feature branches:** - - Workflow triggered by **[push](https://docs.github.com/en/actions/reference/events-that-trigger-workflows#push)** - or any other **[event](https://docs.github.com/en/free-pro-team@latest/actions/reference/events-that-trigger-workflows)** - - The `base` input parameter must not be the same as the branch that triggered the workflow - - Changes are detected against the merge-base with the configured base branch or the default branch - - Uses git commands to detect changes - repository must be already [checked out](https://github.com/actions/checkout) -- **Master, Release, or other long-lived branches:** - - Workflow triggered by **[push](https://docs.github.com/en/actions/reference/events-that-trigger-workflows#push)** event - when `base` input parameter is the same as the branch that triggered the workflow: - - Changes are detected against the most recent commit on the same branch before the push - - Workflow triggered by any other **[event](https://docs.github.com/en/free-pro-team@latest/actions/reference/events-that-trigger-workflows)** - when `base` input parameter is commit SHA: - - Changes are detected against the provided `base` commit - - Workflow triggered by any other **[event](https://docs.github.com/en/free-pro-team@latest/actions/reference/events-that-trigger-workflows)** - when `base` input parameter is the same as the branch that triggered the workflow: - - Changes are detected from the last commit - - Uses git commands to detect changes - repository must be already [checked out](https://github.com/actions/checkout) -- **Local changes** - - Workflow triggered by any event when `base` input parameter is set to `HEAD` - - Changes are detected against the current HEAD - - Untracked files are ignored - -## Example - -```yaml -- uses: dorny/paths-filter@v3 - id: changes - with: - filters: | - src: - - 'src/**' - - # run only if some file in 'src' folder was changed -- if: steps.changes.outputs.src == 'true' - run: ... -``` - -For more scenarios see [examples](#examples) section. - -## Notes - -- Paths expressions are evaluated using [picomatch](https://github.com/micromatch/picomatch) library. - Documentation for path expression format can be found on the project GitHub page. -- Picomatch [dot](https://github.com/micromatch/picomatch#options) option is set to true. - Globbing will also match paths where file or folder name starts with a dot. -- It's recommended to quote your path expressions with `'` or `"`. Otherwise, you will get an error if it starts with `*`. -- Local execution with [act](https://github.com/nektos/act) works only with alternative runner image. Default runner doesn't have `git` binary. - - Use: `act -P ubuntu-latest=nektos/act-environments-ubuntu:18.04` - -## What's New - -- New major release `v3` after update to Node 20 [Breaking change] -- Add `ref` input parameter -- Add `list-files: csv` format -- Configure matrix job to run for each folder with changes using `changes` output -- Improved listing of matching files with `list-files: shell` and `list-files: escape` options -- Paths expressions are now evaluated using [picomatch](https://github.com/micromatch/picomatch) library - -For more information, see [CHANGELOG](https://github.com/dorny/paths-filter/blob/master/CHANGELOG.md) +- Support ignore file and paths by using `ignore` option in [picomatch](https://github.com/micromatch/picomatch) ## Usage -```yaml -- uses: dorny/paths-filter@v3 - with: - # Defines filters applied to detected changed files. - # Each filter has a name and a list of rules. - # Rule is a glob expression - paths of all changed - # files are matched against it. - # Rule can optionally specify if the file - # should be added, modified, or deleted. - # For each filter, there will be a corresponding output variable to - # indicate if there's a changed file matching any of the rules. - # Optionally, there can be a second output variable - # set to list of all files matching the filter. - # Filters can be provided inline as a string (containing valid YAML document), - # or as a relative path to a file (e.g.: .github/filters.yaml). - # Filters syntax is documented by example - see examples section. - filters: '' - - # Branch, tag, or commit SHA against which the changes will be detected. - # If it references the same branch it was pushed to, - # changes are detected against the most recent commit before the push. - # Otherwise, it uses git merge-base to find the best common ancestor between - # current branch (HEAD) and base. - # When merge-base is found, it's used for change detection - only changes - # introduced by the current branch are considered. - # All files are considered as added if there is no common ancestor with - # base branch or no previous commit. - # This option is ignored if action is triggered by pull_request event. - # Default: repository default branch (e.g. master) - base: '' - - # Git reference (e.g. branch name) from which the changes will be detected. - # Useful when workflow can be triggered only on the default branch (e.g. repository_dispatch event) - # but you want to get changes on a different branch. - # This option is ignored if action is triggered by pull_request event. - # default: ${{ github.ref }} - ref: - - # How many commits are initially fetched from the base branch. - # If needed, each subsequent fetch doubles the - # previously requested number of commits until the merge-base - # is found, or there are no more commits in the history. - # This option takes effect only when changes are detected - # using git against base branch (feature branch workflow). - # Default: 100 - initial-fetch-depth: '' - - # Enables listing of files matching the filter: - # 'none' - Disables listing of matching files (default). - # 'csv' - Coma separated list of filenames. - # If needed, it uses double quotes to wrap filename with unsafe characters. - # 'json' - File paths are formatted as JSON array. - # 'shell' - Space delimited list usable as command-line argument list in Linux shell. - # If needed, it uses single or double quotes to wrap filename with unsafe characters. - # 'escape'- Space delimited list usable as command-line argument list in Linux shell. - # Backslash escapes every potentially unsafe character. - # Default: none - list-files: '' - - # Relative path under $GITHUB_WORKSPACE where the repository was checked out. - working-directory: '' - - # Personal access token used to fetch a list of changed files - # from GitHub REST API. - # It's only used if action is triggered by a pull request event. - # GitHub token from workflow context is used as default value. - # If an empty string is provided, the action falls back to detect - # changes using git commands. - # Default: ${{ github.token }} - token: '' - - # Optional parameter to override the default behavior of file matching algorithm. - # By default files that match at least one pattern defined by the filters will be included. - # This parameter allows to override the "at least one pattern" behavior to make it so that - # all of the patterns have to match or otherwise the file is excluded. - # An example scenario where this is useful if you would like to match all - # .ts files in a sub-directory but not .md files. - # The filters below will match markdown files despite the exclusion syntax UNLESS - # you specify 'every' as the predicate-quantifier parameter. When you do that, - # it will only match the .ts files in the subdirectory as expected. - # - # backend: - # - 'pkg/a/b/c/**' - # - '!**/*.jpeg' - # - '!**/*.md' - predicate-quantifier: 'some' -``` - -## Outputs - -- For each filter, it sets output variable named by the filter to the text: - - `'true'` - if **any** of changed files matches any of filter rules - - `'false'` - if **none** of changed files matches any of filter rules -- For each filter, it sets an output variable with the name `${FILTER_NAME}_count` to the count of matching files. -- If enabled, for each filter it sets an output variable with the name `${FILTER_NAME}_files`. It will contain a list of all files matching the filter. -- `changes` - JSON array with names of all filters matching any of the changed files. - -## Examples - -### Conditional execution - -
- Execute step in a workflow job only if some file in a subfolder is changed +A example of usage is as follows: ```yaml jobs: - tests: - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@v4 - - uses: dorny/paths-filter@v3 - id: filter - with: - filters: | - backend: - - 'backend/**' - frontend: - - 'frontend/**' - - # run only if 'backend' files were changed - - name: backend tests - if: steps.filter.outputs.backend == 'true' - run: ... - - # run only if 'frontend' files were changed - - name: frontend tests - if: steps.filter.outputs.frontend == 'true' - run: ... - - # run if 'backend' or 'frontend' files were changed - - name: e2e tests - if: steps.filter.outputs.backend == 'true' || steps.filter.outputs.frontend == 'true' - run: ... -``` - -
- -
- Execute job in a workflow only if some file in a subfolder is changed - -```yml -jobs: - # JOB to run change detection - changes: - runs-on: ubuntu-latest - # Required permissions - permissions: - pull-requests: read - # Set job outputs to values from filter step + check-changes: outputs: - backend: ${{ steps.filter.outputs.backend }} - frontend: ${{ steps.filter.outputs.frontend }} - steps: - # For pull requests it's not necessary to checkout the code - - uses: dorny/paths-filter@v3 - id: filter - with: - filters: | - backend: - - 'backend/**' - frontend: - - 'frontend/**' - - # JOB to build and test backend code - backend: - needs: changes - if: ${{ needs.changes.outputs.backend == 'true' }} - runs-on: ubuntu-latest + run-client: ${{ steps.filter.outputs.client }} + run-server: ${{ steps.filter.outputs.server }} + run-all: ${{ steps.filter.outputs.all }} steps: - uses: actions/checkout@v4 - - ... - - # JOB to build and test frontend code - frontend: - needs: changes - if: ${{ needs.changes.outputs.frontend == 'true' }} - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@v4 - - ... -``` - -
- -
-Use change detection to configure matrix job - -```yaml -jobs: - # JOB to run change detection - changes: - runs-on: ubuntu-latest - # Required permissions - permissions: - pull-requests: read - outputs: - # Expose matched filters as job 'packages' output variable - packages: ${{ steps.filter.outputs.changes }} - steps: - # For pull requests it's not necessary to checkout the code - - uses: dorny/paths-filter@v3 - id: filter - with: - filters: | - package1: src/package1 - package2: src/package2 - - # JOB to build and test each of modified packages - build: - needs: changes - strategy: - matrix: - # Parse JSON array containing names of all filters matching any of changed files - # e.g. ['package1', 'package2'] if both package folders contains changes - package: ${{ fromJSON(needs.changes.outputs.packages) }} - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@v4 - - ... -``` - -
- -### Change detection workflows - -
- Pull requests: Detect changes against PR base branch - -```yaml -on: - pull_request: - branches: # PRs to the following branches will trigger the workflow - - master - - develop -jobs: - build: - runs-on: ubuntu-latest - # Required permissions - permissions: - pull-requests: read - steps: - - uses: actions/checkout@v4 - - uses: dorny/paths-filter@v3 - id: filter - with: - filters: ... # Configure your filters -``` - -
- -
- Feature branch: Detect changes against configured base branch - -```yaml -on: - push: - branches: # Push to following branches will trigger the workflow - - feature/** -jobs: - build: - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@v4 - with: - # This may save additional git fetch roundtrip if - # merge-base is found within latest 20 commits - fetch-depth: 20 - - uses: dorny/paths-filter@v3 - id: filter - with: - base: develop # Change detection against merge-base with this branch - filters: ... # Configure your filters -``` - -
- -
- Long lived branches: Detect changes against the most recent commit on the same branch before the push - -```yaml -on: - push: - branches: # Push to the following branches will trigger the workflow - - master - - develop - - release/** -jobs: - build: - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@v4 - - uses: dorny/paths-filter@v3 - id: filter - with: - # Use context to get the branch where commits were pushed. - # If there is only one long-lived branch (e.g. master), - # you can specify it directly. - # If it's not configured, the repository default branch is used. - base: ${{ github.ref }} - filters: ... # Configure your filters -``` - -
- -
- Local changes: Detect staged and unstaged local changes - -```yaml -on: - push: - branches: # Push to following branches will trigger the workflow - - master - - develop - - release/** -jobs: - build: - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@v4 - - # Some action that modifies files tracked by git (e.g. code linter) - - uses: johndoe/some-action@v1 - - # Filter to detect which files were modified - # Changes could be, for example, automatically committed - - uses: dorny/paths-filter@v3 - id: filter - with: - base: HEAD - filters: ... # Configure your filters + - uses: gylove1994/paths-filter@v2 + id: filter + with: + filters: | + client: + pattern: + - 'src/**' + - 'tsconfig.src.json' + - 'vite.config.ts' + server: + pattern: + - 'srv/**' + - 'prisma/**' + - 'tsconfig.srv.json' + - 'tsconfig.srv.build.json' + - 'nest-cli.json' + all: + pattern: + - '**' + - '*' + ignore: + - 'src/**' + - 'srv/**' + - 'prisma/**' + - 'nest-cli.json' + - 'tsconfig.src.json' + - 'vite.config.ts' + - 'tsconfig.srv.json' + - 'tsconfig.srv.build.json' ``` -
- -### Advanced options - -
- Define filter rules in own file - -```yaml -- uses: dorny/paths-filter@v3 - id: filter - with: - # Path to file where filters are defined - filters: .github/filters.yaml -``` - -
- -
- Use YAML anchors to reuse path expression(s) inside another rule - -```yaml -- uses: dorny/paths-filter@v3 - id: filter - with: - # &shared is YAML anchor, - # *shared references previously defined anchor - # src filter will match any path under common, config and src folders - filters: | - shared: &shared - - common/** - - config/** - src: - - *shared - - src/** -``` - -
- -
- Consider if file was added, modified or deleted - -```yaml -- uses: dorny/paths-filter@v3 - id: filter - with: - # Changed file can be 'added', 'modified', or 'deleted'. - # By default, the type of change is not considered. - # Optionally, it's possible to specify it using nested - # dictionary, where the type of change composes the key. - # Multiple change types can be specified using `|` as the delimiter. - filters: | - shared: &shared - - common/** - - config/** - addedOrModified: - - added|modified: '**' - allChanges: - - added|deleted|modified: '**' - addedOrModifiedAnchors: - - added|modified: *shared -``` - -
- -
- Detect changes in folder only for some file extensions - -```yaml -- uses: dorny/paths-filter@v3 - id: filter - with: - # This makes it so that all the patterns have to match a file for it to be - # considered changed. Because we have the exclusions for .jpeg and .md files - # the end result is that if those files are changed they will be ignored - # because they don't match the respective rules excluding them. - # - # This can be leveraged to ensure that you only build & test software changes - # that have real impact on the behavior of the code, e.g. you can set up your - # build to run when Typescript/Rust/etc. files are changed but markdown - # changes in the diff will be ignored and you consume less resources to build. - predicate-quantifier: 'every' - filters: | - backend: - - 'pkg/a/b/c/**' - - '!**/*.jpeg' - - '!**/*.md' -``` - -
- -### Custom processing of changed files - -
- Passing list of modified files as command line args in Linux shell - -```yaml -- uses: dorny/paths-filter@v3 - id: filter - with: - # Enable listing of files matching each filter. - # Paths to files will be available in `${FILTER_NAME}_files` output variable. - # Paths will be escaped and space-delimited. - # Output is usable as command-line argument list in Linux shell - list-files: shell - - # In this example changed files will be checked by linter. - # It doesn't make sense to lint deleted files. - # Therefore we specify we are only interested in added or modified files. - filters: | - markdown: - - added|modified: '*.md' -- name: Lint Markdown - if: ${{ steps.filter.outputs.markdown == 'true' }} - run: npx textlint ${{ steps.filter.outputs.markdown_files }} -``` - -
- -
- Passing list of modified files as JSON array to another action - -```yaml -- uses: dorny/paths-filter@v3 - id: filter - with: - # Enable listing of files matching each filter. - # Paths to files will be available in `${FILTER_NAME}_files` output variable. - # Paths will be formatted as JSON array - list-files: json - - # In this example all changed files are passed to the following action to do - # some custom processing. - filters: | - changed: - - '**' -- name: Lint Markdown - uses: johndoe/some-action@v1 - with: - files: ${{ steps.filter.outputs.changed_files }} -``` - -
- -## See also - -- [test-reporter](https://github.com/dorny/test-reporter) - Displays test results from popular testing frameworks directly in GitHub - ## License The scripts and documentation in this project are released under the [MIT License](https://github.com/dorny/paths-filter/blob/master/LICENSE)