User Supplied Exclude Filter

This filter allows users to specify optional custom regular expression (regex) patterns to exclude certain files from being processed by the program's operations: annotate, doc, embed, explain, implement, report, and misc.

Filter Structure

The exclude filter is defined using a JSON file containing an array of strings, where each entry is a regex pattern that matches file paths to be excluded from processing. This structure allows for straightforward customization and scalability, enabling users to manage large sets of exclusion rules efficiently.

If the filter file cannot be read, cannot be decoded as a JSON string array, or contains an invalid regex pattern, the operation will stop with an error.

Example

[
    "(?i)^vendor(\\\\|\\/).*",
    "other\\/.*\\.go$"
]

Key Points

• Regex Patterns: Utilize standard Go regex syntax to define patterns. Patterns are compiled with Go's regexp package and are applied with substring matching, so use anchors when you need precise path matching.
• Case Sensitivity: Regex matching is case-sensitive by default. To make a regex case-insensitive, add (?i) to the beginning of the regex.
• Directory Separator: Directory separators are platform-specific. To ensure your filter works on any platform, include both path separators in your regex.
• Special Character Escaping: Note that the \ character has a special meaning in regexps. The / character does not need to be escaped in Go, but you may still want to use it for compatibility with other regex engines. To write a \ character in a JSON string, you need to escape it with another \. For example, to pass \\ to a Go regex, you need to specify it as \\\\ inside a JSON string. Therefore, a regex group that matches path separators on any platform can be written in the JSON string as (\\\\|\\/).
• Relative Path: File paths in Perpetual are passed as paths relative to the project root; they begin from the project root.
• Anchors: Use ^ and $ to denote the start and end of a path for precise matching.
• Usage: The filter is applied via the -x flag in supported operations, which specifies the path to the JSON file containing the exclude patterns. The filter works in conjunction with project-specific whitelist and blacklist configurations. It can only exclude files; it does not add files that are not selected by the project whitelist. Depending on the operation, the user filter may be appended to the project blacklist before collecting files, or applied later to the operation-specific list of files to process.
• Operation Behavior: In doc, explain, implement, and report, the filter is added to the project blacklist before the project file list is built. In misc, it is applied after the project file list is collected for file-oriented modes such as listing or checking files (-m list, -m check-read, -m check-ascii, -m save-utf); it has no effect on the misc project test mode (-m proj-test), which returns before the file list is filtered. In annotate and embed, it filters the files selected for annotation or embedding; skipped files keep their previous stored checksum state so they can be processed again later if the filter is removed. In embed question/search mode (-m query), the filter is also applied to the file list used for similarity search.