Troubleshooting

August 26, 2026 · View on GitHub

Common problems

Permission denied

Almost every command needs root.

sudo solo-provisioner block node install --profile=local

Exceptions: solo-provisioner version, and block node reconcile-shaper --check.

`profile flag is required`

Every block node command needs --profile.

sudo solo-provisioner block node check --profile=mainnet

kube cluster install is the exception — it takes no profile. See Deployment profiles.

`invalid base path`

The storage path must exist and be writable.

sudo mkdir -p /mnt/storage
sudo solo-provisioner block node install --profile=mainnet --base-path=/mnt/storage

To change a storage path on an existing install you also need --purge-storage — a local PV's hostPath.path is immutable. See reconfigure.

Helm chart problems

Pin the version explicitly instead of resolving the latest:

sudo solo-provisioner block node install --profile=mainnet --chart-version=0.22.1

If the install times out, raise --timeout. The default is 5m0s, and exceeding it rolls the operation back.

Alloy fails: cluster not reachable

Alloy deploys into an existing cluster; it never creates one. Install the cluster first:

sudo solo-provisioner kube cluster install

Locked out of SSH after enabling the firewall

The host firewall is default-drop. An empty management allowlist admits nobody.

block node install --firewall-enabled with an empty --mgmt-cidrs skips the firewall on purpose, exactly to avoid this.
network firewall create with no --mgmt-cidrs does not skip it — it renders an empty allowlist. Always pass --mgmt-cidrs.

Recovery needs console access. Then:

sudo solo-provisioner network firewall add --name mgmt --cidr <your-cidr>

Corrupt firewall config

Every apply keeps the previous generation. Restore it and re-apply:

sudo cp /etc/solo-provisioner/network-weaver-host-firewall.yaml.prev \
        /etc/solo-provisioner/network-weaver-host-firewall.yaml
sudo solo-provisioner network firewall reapply

Full detail: Recovering a corrupt config.

Traffic shaping is on but nothing is being shaped

Check in this order:

Is the daemon running?

sudo solo-provisioner daemon service check

Do the policies exist, and do their sets have members?
```
sudo solo-provisioner network policy show
```
Sets start empty and are filled by the daemon from the block node's statusz. An empty set has two different causes, and they are not the same problem:
- Statusz has not answered yet — first boot, or a reboot before the first poll. nft set membership is not boot-persistent, so it is rebuilt on the first successful poll.
- Statusz answered 200 with nothing in it. A successful poll that reports no endpoints for a category clears that category's set.
A poll that fails (statusz unreachable, or an apply error) does not empty anything — it leaves the last-good membership in place and retries on the next tick. So an empty set in steady state points at an empty statusz response, not at a connectivity problem. Check the endpoint the daemon is actually polling with --statusz-base-url, then read it yourself:
```
solo-provisioner block node reconcile-shaper --statusz-url=<url> --check
```
--check needs no privilege and touches no nft state — it just prints what the daemon would apply. See What an empty statusz response does.
Is traffic actually landing in a class?
```
sudo solo-provisioner network shape watch --device egress --iface enp0s1
```
A non-zero rate against a class means classification is working.
Is ingress shaping attached to the pod? Ingress lives on the pod's host-side veth and is attached per-pod by the daemon. Find the veth with ip link, then:
```
sudo solo-provisioner network shape watch --device ingress --iface lxc1a2b3c
```

Rollback did not seem to run

--rollback-on-error is not shown in the TUI. Confirm it another way:

helm list -A

or read the workflow report YAML whose path is printed as report_path=….

Getting more detail

Expanded output

sudo solo-provisioner block node install --profile=local --verbose

Debug logs

sudo solo-provisioner block node install --profile=local --log-level=debug

or in the config file:

log:
  level: debug
  consoleLogging: true

Raw logs, no TUI

Useful in CI, or when the TUI is hiding something:

sudo solo-provisioner block node install --profile=local --non-interactive

Machine-readable output

sudo solo-provisioner block node install --profile=local -o json \
  | jq 'select(.type=="summary")'

Help

solo-provisioner --help
solo-provisioner block --help
solo-provisioner block node --help
solo-provisioner block node install --help

Still stuck? Open an issue at hashgraph/solo-weaver, and include the workflow report YAML named in the report_path=… line.