The Broker config file tells Broker about the repositories it should scan, how it can access them, and at what cadence.

Quick Start

For more detail, see the sections below. The format is as follows:

fossa_endpoint: https://app.fossa.com
fossa_integration_key: abcd1234
version: 1
concurrency: 10

debugging:
  location: /home/me/.config/fossa/broker/debugging/
  retention:
    days: 7

integrations:
- type: git
  poll_interval: 1h
  url: git@github.com:fossas/broker.git
  auth:
    type: ssh_key_file
    path: /home/me/.ssh/id_rsa

Version

The config file is versioned. At this time, the only supported version is 1. It is required to have version present in the config file.

Concurrency

Configures the global concurrency of configured integrations. The default value, if not specified, is 10. If a negative value or 0 is specified, also defaults to 10.

When this is configured, poll and fetch operations for configured integrations are limited to the specified number of operations running at any given moment.

The maximum value for this field is 2147483647 (int32 max), but you almost definitely want something lower even on the most powerful of servers.

FOSSA communication

Value	Required?	Description	Suggested default
fossa_endpoint	Required	The address to the FOSSA instance.	https://app.fossa.com
fossa_integration_key	Required	The API key for FOSSA.	N/A

FOSSA integration keys can be created at Settings → Integrations → API.

The existing level of functionality will always be supported using a "push-only" key, but future features may require a "full" key to get the most use.

Debugging

This block specifies where Broker stores its debugging artifacts. For more information on what a "debugging artifact" is, see Debug Artifacts.

Value	Required?	Description	Suggested default
location	Required	The root directory into which debug artifacts are written.	{USER_HOME}/.config/fossa/broker/debugging/
retention.days	Optional	Remove debug artifacts that are older than this time span.	7

Integrations

Broker can be configured to integrate with multiple code hosts using this configuration block. This is an array of blocks, specified by type.

Supported types:

Type	Description
git	A remote git repository
gitlab_group	Every repository in a GitLab group, discovered automatically

git

This block specifies how to configure Broker to communicate with a git server for a specific git repository.

Value	Required?	Description	Suggested default	Minimum value
poll_interval	Required	How often Broker checks with the remote repository to see whether it has changed.1	1 hour	1 hour
remote	Required	The remote git repository address.	N/A	N/A
auth	Required	Required authentication to clone this repository.	N/A	N/A
team	Optional	The team in FOSSA to which this project should be assigned.2	N/A	N/A
title	Optional	Specify a custom title for the project instead of using the default.3	N/A	N/A
labels	Optional	Labels to apply to the project in FOSSA.4	N/A	N/A
import_branches	Optional	Initialize to scan specific branches for the remote repository	N/A	N/A
import_tags	Optional	Initialize to scan tags for the remote repository	N/A	N/A
watched_branches	Optional	The name of the branches that you intend to scan	N/A	N/A

[1]: The poll interval defines the interval at which Broker checks for updates, not the interval at which Broker actually analyzes the repository. For more details on authentication, see integration authentication.

[2]: Team settings only affect newly imported projects. Changing this value later requires using the FOSSA UI. If the project already exists before transitioning it to be managed by Broker, this also has no effect.

[3]: Title settings only affect newly imported projects. Changing this value later requires using the FOSSA UI. If the project already exists before transitioning it to be managed by Broker, this also has no effect. If unspecified, Broker uses a default title, which is just the configured git remote.

[4]: Labels are applied on every upload, so adding a label to an existing integration applies it on the next scan. Labels that do not already exist in your FOSSA organization are created automatically. If a label cannot be created or applied — for example because the organization's label limit is reached — the upload still succeeds and the failure is reported as a build warning.

gitlab_group

This block scans every repository in a GitLab group without listing them individually.

Broker asks GitLab which repositories the group contains and expands the block into one git integration per repository. This is intended for groups large enough that enumerating repositories by hand is impractical.

Value	Required?	Description	Suggested default
poll_interval	Required	How often Broker checks each discovered repository for changes.1	1 hour
group	Required	The group's full path. May be a subgroup, for example parent/child.	N/A
auth	Required	Credential used both to discover repositories and to clone them.2	N/A
host	Optional	The GitLab instance. Set this for self-managed GitLab.	https://gitlab.com
include_subgroups	Optional	Whether to include repositories in subgroups.	true
team	Optional	The team in FOSSA to assign every discovered project to.	N/A
labels	Optional	Labels applied to every discovered project. See the git integration's note.	N/A
import_branches	Optional	Applied to every discovered repository.	N/A
import_tags	Optional	Applied to every discovered repository.	N/A
watched_branches	Optional	Applied to every discovered repository.3	N/A

[1]: The poll interval applies to each discovered repository independently, and governs how often Broker checks an already-discovered repository for new commits. It does not control how often Broker looks for new repositories — see Discovery runs at startup below.

Be deliberate about this value for large groups: a group of several thousand repositories polled hourly is a substantial and continuous load on both Broker and the GitLab instance. Note that the concurrency setting is ignored in this release (see the changelog), so it cannot currently be used to bound that load.

[2]: Only http_basic and http_header are supported here, because discovery calls the GitLab API and needs a credential it can send as an HTTP header. See integration authentication.

[3]: If watched_branches is not set, Broker scans the branch GitLab reports as each repository's default. Unlike the git integration, Broker does not need to contact each repository to infer this, because GitLab provides it during discovery.

Broker skips repositories that are archived, and repositories with no default branch (which usually means they have no commits). Each skipped repository is named in the logs. Repositories shared into the group from elsewhere are not included; only repositories the group owns are scanned.

Discovered projects are titled with their path within GitLab, for example my-org/platform/api.

Discovery runs at startup

Broker discovers the group's repositories once, when it starts.

Changes within an already-discovered repository are picked up continuously, on the configured poll_interval. But repositories added to the group after Broker started are not scanned until Broker is restarted, and repositories removed from the group continue to be polled until then.

If repositories are added to your group regularly, run Broker under a supervisor that restarts it periodically — for example a systemd service with a daily restart, or a launchd job on macOS. Discovery re-runs on each start, so a restart is all that is needed to pick up everything new. A daily cadence is sufficient for most groups.

For a group whose membership rarely changes, restarting Broker when you know repositories were added is enough.

Using a GitLab group access token

A group access token is the most direct way to configure this block, because one credential covers every repository in the group. Pass it as the password of an http_basic block:

integrations:
- type: gitlab_group
  poll_interval: 1h
  group: your-group
  team: Platform
  labels:
    - gitlab
    - imported-by-broker
  auth:
    type: http_basic
    username: fossa-broker      # the token's name; GitLab ignores the username
    password: glpat-xxxxxxxxxxxxxxxxxxxx

Because Broker polls rather than receiving webhooks, this token does not need write access. The read_api and read_repository scopes are sufficient: read_api to discover repositories and read_repository to clone them.

For self-managed GitLab, also set host:

- type: gitlab_group
  poll_interval: 1h
  host: https://gitlab.example.com
  group: your-group
  auth:
    type: http_basic
    username: fossa-broker
    password: glpat-xxxxxxxxxxxxxxxxxxxx

Appendix

duration values

A duration is made up of {value}{unit} pairs, where value is the count of units. For example, the input 5h 30min means "5 hours and 30 minutes". If a single value is provided with no time unit, it is assumed to be seconds.

To specify a time unit, use one of the below forms:

• Nanoseconds: nsec, ns
• Microseconds: usec, us
• Milliseconds: msec, ms
• Seconds: seconds, second, sec, s
• Minutes: minutes, minute, min, m
• Hours: hours, hour, hr, h
• Days: days, day, d
• Weeks: weeks, week, w
• Months: months, month, M
• Years: years, year, y

Examples for valid durations:

Input	Description
2h	2 hours
2hours	2 hours
48hr	48 hours
1y 12month	1 year and 12 months
55s 500ms	55 seconds and 500 milliseconds
300ms 20s 5day	5 days, 20 seconds, and 300 milliseconds
5day 4hours 10days	15 days and 4 hours

Smart Imports

Broker provides configurable branch/tag scanning for every integration. You can customize your scans through these fields listed in the integrations section of your config.yml:

integrations:
  - type: git
    import_branches: true  # Defaults to true
    watched_branches:      # If unspecified, Broker will try to set to main or master if present
      - main
      - release*
    import_tags: false     # Defaults to false

default values

If these fields are not set, import_branches will be set to true, import_tags will be set to false, and Broker will make a best effort approach to set watched_branches to main or master if it is present in the remote.

branch scanning

In order to scan specific branches, import_branches must be set to true and the list of branches you intend to scan should be provided under watched_branches. Having watched_branches set while having import_branches set to false is an invalid combination and will cause Broker to throw errors.

Glob matching is also provided with your branches. If one of your watched_branches is release* and your remote contains branches release1, release2, and release-3. Then all three of those branches will be scanned due to glob matching.

tag scanning

In order to allow Broker to scan tags in your remote, import_tags must be set to true

toggling fields

Toggling import_branches from true to false will remove all existing uploaded scans for ALL branches of that particular remote in your local database (this does NOT delete your scans in the FOSSA UI). If toggled from false to true, Broker will perform as if it is scanning the listed watched_branches for the first time. On subsequent poll cycles, Broker will import the latest changes from your configured branches since the last revision (skipping any intermediate commits).

Toggling import_tags from true to false will remove all existing uploaded scans for ALL tags of that particular remote in your local database (this does NOT delete your scans in the FOSSA UI). If toggled from false to true, Broker will perform as if it is scanning all the remote's tags for the first time. This would mean that all tags for that remote would be scanned. On subsequent poll cycles, Broker will import all created or changed tags since the last poll cycle.

Integration authentication

Integrations support several possible authentication schemes, specified by type. Which authentication method used mostly depends on your specific git server and the URL provided in the integration.

If the url begins with http:// or https://, valid authentication types are http_basic or http_header. If the url begins with ssh://, valid authentication types are ssh_key or ssh_key_file.

Security: Broker assumes the local file system is trusted. While it does its best to ensure secrets exist on disk for the minimum time possible, it may write secrets to the temporary directory during the course of its operation. On unix-based operating systems, the temporary directory location may be specified with the TMPDIR environment variable.

none

If no authentication is required, specify type "none". This requires a "transport" field, so that Broker can determine which transport mechanism (HTTP or SSH) to use to clone the repository. Usually this is determined automatically by the authentication type, but in this case it has to be manually specified.

Example integration block:

- type: git
  poll_interval: 1h
  remote: https://github.com/fossas/broker.git
  auth:
    type: none
    transport: http

http_basic

Performs authentication with a username and password. Example integration block:

- type: git
  poll_interval: 1h
  remote: https://github.com/fossas/broker.git
  auth:
    type: http_basic
    username: jssblck
    password: abcd1234

http_header

Performs authentication with a constant header. Example integration block:

- type: git
  poll_interval: 1h
  remote: https://github.com/fossas/broker.git
  auth:
    type: http_header
    header: "Authorization: Bearer abcd1234"

ssh_key

Performs authentication with a constant SSH private key. Example integration block:

- type: git
  poll_interval: 1h
  remote: git@github.com:fossas/broker.git
  auth:
    type: ssh_key
    key: |
      -----BEGIN OPENSSH PRIVATE KEY-----
      key goes here
      -----END OPENSSH PRIVATE KEY-----

ssh_key_file

Performs authentication with an SSH private key file. Example integration block:

- type: git
  poll_interval: 1h
  remote: git@github.com:fossas/broker.git
  auth:
    type: ssh_key_file
    path: /home/me/.ssh/id_rsa