Adaptors

The adaptor registry catalogues adaptors, versions, credential schemas and icons for the workflow editor. Lightning fetches it from npm by default, cached in Postgres. For the Elixir side, start at Lightning.Adaptors.

Using local adaptors

Point Lightning at a checkout of the adaptors monorepo, not npm:

ADAPTORS_STRATEGY=local
ADAPTORS_LOCAL_REPO=/path/to/adaptors

The path is the repo root, not packages/. Each subdirectory with a package.json becomes an adaptor, named and versioned from it. Lightning also reads configuration-schema.json for the credential form and assets/square/assets/rectangle (.png/.svg) for icons; a package without them still appears, minus the form or icon.

To layer a private checkout over the public one, comma-separate multiple roots:

ADAPTORS_LOCAL_REPO=/path/to/private-adaptors,/path/to/adaptors

A package in multiple roots comes from the first. Lightning logs each shadowed package on every scan.

Note {: .info}

The deprecated LOCAL_ADAPTORS=true and OPENFN_ADAPTORS_REPO still work. Lightning warns at boot only when it falls back to them: LOCAL_ADAPTORS=true when ADAPTORS_STRATEGY is unset, OPENFN_ADAPTORS_REPO when the strategy is local and ADAPTORS_LOCAL_REPO is unset.

Running without internet access

On an internet-connected, refreshed instance, dump the catalogue. The dump holds icon metadata only, so archive the icons directory too:

mix lightning.adaptors.dump --path snapshot.json
tar czf icons.tar.gz -C "$ADAPTORS_ICONS_PATH" .

ADAPTORS_ICONS_PATH is /app/priv/adaptor_icons in the official image, and otherwise defaults to lightning/adaptor_icons under the temp directory. On a release image (no Mix), dump with:

bin/lightning eval 'Lightning.Release.dump_adaptors("/path/to/snapshot.json")'

Offline, unpack icons to ADAPTORS_ICONS_PATH, then import:

mkdir -p "$ADAPTORS_ICONS_PATH"
tar xzf icons.tar.gz -C "$ADAPTORS_ICONS_PATH"
mix lightning.adaptors.import --path snapshot.json --replace

On a release image, import with:

bin/lightning eval 'Lightning.Release.seed_adaptors("/path/to/snapshot.json", replace: true)'

With no populated instance, build the snapshot from npm anywhere online (no database, no icons):

mix lightning.adaptors.snapshot --path snapshot.json

Import as above.

Internal mirrors: any npm-compatible registry works. Set ADAPTORS_NPM_REGISTRY_URL, ADAPTORS_NPM_JSDELIVR_URL and ADAPTORS_NPM_GITHUB_URL, leave the strategy as npm, and set ADAPTORS_NPM_GITHUB_REF if the mirror serves a branch other than main.

A registry that answers but lists no @openfn/language-* packages counts as a failed listing: almost always a mistyped mirror URL, or a mirror that has not synced the @openfn scope. The rows already in Postgres stay as they are and the next refresh tries again.

An imported catalogue survives the hourly refresh; a failed one logs a warning and leaves rows alone.

The worker installs adaptor packages into ADAPTORS_PATH itself, a separate download not covered here.

Keeping the catalogue fresh

Lightning refreshes the catalogue hourly. Set ADAPTORS_REFRESH_INTERVAL_SECONDS to change that interval, or to 0 to disable scheduled refreshes. Force one manually, on a source checkout:

mix lightning.adaptors.refresh
mix lightning.adaptors.refresh --name @openfn/language-http

Without --name it runs a full refresh and waits; with --name it refetches that adaptor regardless of version change. A cycle that ran but wrote no rows exits 0; a source that could not be listed at all exits 2. The full list is in mix help lightning.adaptors.refresh.

A release image has no Mix; run the same call against the node:

bin/lightning rpc 'Lightning.Adaptors.refresh(await: true)'
bin/lightning rpc 'Lightning.Adaptors.refresh_package("@openfn/language-http")'

While the catalogue is still loading

A fresh instance has an empty catalogue until the first refresh lands. Reads do not wait for it: the adaptor picker and the credential form show "Couldn't load adaptors" with a Retry button, and GET /adaptors/catalogue replies 503 with a retry-after header. All three recover on their own once the load lands.

Saving a workflow is the exception. It waits for the first load, up to 90 seconds, and rejects the save with "adaptor catalogue is not ready yet" if nothing has arrived. So an instance that cannot reach npm serves every page but cannot save a workflow until the catalogue has loaded once. Import a snapshot to give it one.

Troubleshooting

• Adaptor missing from the picker: find its fetch_adaptor warning in the log, then force a refresh with --name.
• New version not showing: the hourly refresh hasn't run. Force one, or wait.
• Icons missing after import: they never reached ADAPTORS_ICONS_PATH on this instance. Unpack the icons archive there, or redo the dump and copy the directory.
• Local package ignored: an earlier ADAPTORS_LOCAL_REPO root has a package of the same name; the log names each shadowed package.
• Deprecated-variable boot warning: rename LOCAL_ADAPTORS=true to ADAPTORS_STRATEGY=local and OPENFN_ADAPTORS_REPO to ADAPTORS_LOCAL_REPO.
• Picker stuck on "Couldn't load adaptors", or the catalogue endpoint answering 503: the first load has not finished. See While the catalogue is still loading.
• Workflow save rejected with "adaptor catalogue is not ready yet": the same thing, after the save has waited out the first load. If the instance cannot reach npm, import a snapshot; see Running without internet access.
• Refresh exiting 2 with :empty_listing: the registry answered but served no @openfn packages. Check ADAPTORS_NPM_REGISTRY_URL.