Fleets

Before submitting runs, you must create a fleet. Fleets act as both pools of instances and templates for how those instances are provisioned.

dstack supports two fleet types: backend fleets (which are provisioned dynamically in the cloud or on Kubernetes), and SSH fleets (which use existing on-prem servers).

Apply a configuration

To create a fleet, define its configuration in a YAML file. The filename must end with .dstack.yml (e.g. .dstack.yml or fleet.dstack.yml), regardless of fleet type.

=== "Backend fleets" If you're using cloud providers or Kubernetes clusters and have configured the corresponding backends, create a backend fleet as follows:

<div editor-title="fleet.dstack.yml"> 

```yaml
type: fleet
name: my-fleet

# Allow to provision of up to 2 instances
nodes: 0..2

# Uncomment to ensure instances are inter-connected
#placement: cluster

# Deprovision instances above the minimum if they remain idle
idle_duration: 1h

resources:
  # Allow to provision up to 8 GPUs
  gpu: 0..8
```

</div>

Pass the fleet configuration to `dstack apply`:

<div class="termy">

```shell
$ dstack apply -f fleet.dstack.yml
    
  #  BACKEND  REGION           RESOURCES                 SPOT  PRICE
  1  gcp      us-west4         2xCPU, 8GB, 100GB (disk)  yes   \$0.010052
  2  azure    westeurope       2xCPU, 8GB, 100GB (disk)  yes   \$0.0132
  3  gcp      europe-central2  2xCPU, 8GB, 100GB (disk)  yes   \$0.013248

Create the fleet? [y/n]: y

  FLEET     INSTANCE  BACKEND              GPU             PRICE    STATUS  CREATED 
  my-fleet  0         gcp (europe-west-1)  L4:24GB (spot)  \$0.1624  idle    3 mins ago      
            1         gcp (europe-west-1)  L4:24GB (spot)  \$0.1624  idle    3 mins ago    
```

</div>

If the `nodes` range starts with `0`, `dstack apply` creates only a template. Instances are provisioned only when you submit runs.

=== "SSH fleets" If you have a group of on-prem servers accessible via SSH, you can create an SSH fleet as follows:

<div editor-title="fleet.dstack.yml"> 

```yaml
type: fleet
name: my-fleet

# Uncomment if instances are interconnected
#placement: cluster

ssh_config:
  user: ubuntu
  identity_file: ~/.ssh/id_rsa
  hosts:
    - 3.255.177.51
    - 3.255.177.52
```
  
</div>

Pass the fleet configuration to `dstack apply`:

<div class="termy">

```shell
$ dstack apply -f fleet.dstack.yml
    
Provisioning...
---> 100%

  FLEET     INSTANCE  BACKEND       GPU      PRICE  STATUS  CREATED 
  my-fleet  0         ssh (remote)  L4:24GB  \$0     idle    3 mins ago      
            1         ssh (remote)  L4:24GB  \$0     idle    3 mins ago    
```

</div>

`dstack apply` automatically connects to on-prem servers, installs the required dependencies, and adds them to the created fleet.

??? info "Host requirements"
    1.&nbsp;Hosts must be Linux-based and have Docker pre-installed.

    === "NVIDIA"
        2.&nbsp;Hosts with NVIDIA GPUs must also be pre-installed with CUDA 12.1 and
        [NVIDIA Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html).

    === "AMD"
        2.&nbsp;Hosts with AMD GPUs must also be pre-installed with AMDGPU-DKMS kernel driver (e.g. via
        [native package manager](https://rocm.docs.amd.com/projects/install-on-linux/en/latest/install/native-install/index.html)
        or [AMDGPU installer](https://rocm.docs.amd.com/projects/install-on-linux/en/latest/install/amdgpu-install.html).)

    === "Tenstorrent"
        2.&nbsp;Hosts with Tenstorrent accelerators must be pre-installed with [Tenstorrent software](https://docs.tenstorrent.com/getting-started/README.html#software-installation).
        This must include the drivers and HugePages.

    3.&nbsp;The user specified must have passwordless `sudo` access.

    4.&nbsp;The SSH server must be running and configured with `AllowTcpForwarding yes` in `/etc/ssh/sshd_config`.

    5.&nbsp;The firewall must allow SSH and should forbid any other connections from external networks.

    6.&nbsp;If `placement` is set to `cluster`, hosts must be able to communicate with each other.

Once the fleet is created, you can run dev environments, tasks, and services.

Configuration options

Backend fleets support many options; see some major configuration examples below.

Cluster placement

Both backend fleets and SSH fleets allow the placement property to be set to cluster.

This property ensures that instances are interconnected. This is required for running distributed tasks.

=== "Backend fleets" Backend fleets allow to provision interconnected clusters across supported backends.

<div editor-title="fleet.dstack.yml">
    
```yaml
type: fleet
name: my-fleet

nodes: 2
placement: cluster

resources:
  gpu: H100:8
```
    
</div>

#### Backends

Fast interconnect is supported on the `aws`, `gcp`, `nebius`, `crusoe`, and `kubernetes` backends. Some backends may require additional configuration.

=== "GCP"
    On GCP, you may need to configure `extra_vpcs` and `roce_vpcs` in the `gcp` backend configuration.
    Refer to the [GCP](../examples/clusters/gcp.md) examples for more details.

=== "Nebius"
    On [Nebius](https://docs.nebius.com/compute/clusters/gpu), `dstack` automatically configures InfiniBand networking if it is supported by the selected instance type.

=== "Crusoe"
    On [Crusoe](https://docs.crusoecloud.com/networking/infiniband/managing-infiniband-networks), `dstack` automatically configures InfiniBand networking if it is supported by the selected instance type.
    Refer to the [Crusoe](../examples/clusters/crusoe.md#vms) example for more details.

=== "Kubernetes"
    If the Kubernetes cluster has interconnect configured, `dstack` can use it without additional setup.
    See the [Lambda](../examples/clusters/lambda.md#kubernetes) or [Crusoe](../examples/clusters/crusoe.md#kubernetes) examples.

<!-- TODO: Add Runpod tab to cluster backends once cluster support is fully tested -->

=== "SSH fleets" If the hosts in the SSH fleet have interconnect configured, you only need to set placement to cluster.

<div editor-title="fleet.dstack.yml"> 
    
```yaml
type: fleet
name: my-fleet

placement: cluster

ssh_config:
  user: ubuntu
  identity_file: ~/.ssh/id_rsa
  hosts:
    - 3.255.177.51
    - 3.255.177.52
```
  
</div>

!!! info "Network"
    By default, `dstack` automatically detects the network shared by the hosts. However, it's possible to configure it explicitly via the [`network`](../reference/dstack.yml/fleet.md#network) property.

    <!-- TODO: Add network configuration example -->

!!! info "Examples" See the cluster examples for AWS, GCP, Lambda, Crusoe, Nebius, and NCCL/RCCL tests.

Nodes

The nodes property is supported only by backend fleets and specifies how many nodes dstack must or can provision.

type: fleet
name: my-fleet

# Allow to provision of up to 2 instances
nodes: 0..2

# Uncomment to ensure instances are inter-connected
#placement: cluster

# Deprovision instances above the minimum if they remain idle
idle_duration: 1h

resources:
  # Allow to provision up to 8 GPUs
  gpu: 0..8

Pre-provisioning

If the nodes range starts with 0, dstack apply creates only a template, and instances are provisioned when you submit runs.

To provision instances up front, set the nodes range to start above 0. This pre-creates the initial number of instances; additional instances (if any) are provisioned on demand.

```yaml
type: fleet
name: my-fleet

nodes: 2..10

# Uncomment to ensure instances are inter-connected
#placement: cluster

resources:
  gpu: H100:8
```
    
</div>

<div editor-title="fleet.dstack.yml"> 

```yaml
type: fleet
name: my-fleet

nodes:
  min: 2
  max: 10
  target: 6

# Deprovision instances above the minimum if they remain idle
idle_duration: 1h
```

</div>

`dstack apply` pre-provisions up to `target` and scales back to `min` after `idle_duration`.

type: fleet
name: my-fleet

nodes: 0..2

# Uncomment to ensure instances are inter-connected
#placement: cluster

# Allows both on-demand and spot
spot_policy: auto

idle_duration: 1h

resources:
  gpu: 0..8

type: fleet
name: my-fleet

nodes: 2

# Terminate instances idle for more than 1 hour
idle_duration: 1h

resources:
  gpu: 24GB

```yaml
type: fleet
name: my-fleet

nodes: 0..2

resources:
  gpu: H100:8

# Split into 4 blocks, each with 2 GPUs
blocks: 4
```

</div>

```yaml
type: fleet
name: my-fleet

ssh_config:
  user: ubuntu
  identity_file: ~/.ssh/id_rsa
  hosts:
    - hostname: 3.255.177.51
      blocks: 4
    - hostname: 3.255.177.52
      # As many as possible, according to numbers of GPUs and CPUs
      blocks: auto
    - hostname: 3.255.177.53
      # Do not slice. This is the default value, may be omitted
      blocks: 1
```

</div>

```yaml
type: fleet
name: my-fleet

ssh_config:
  user: ubuntu
  identity_file: ~/.ssh/worker_node_key
  hosts:
    - 3.255.177.51
    - 3.255.177.52
  proxy_jump:
    hostname: 3.255.177.50
    user: ubuntu
    identity_file: ~/.ssh/head_node_key
```

```
Host 3.255.177.50
    IdentityFile ~/.ssh/head_node_key
```

type: fleet
name: my-fleet

env:
  - HTTP_PROXY=http://proxy.example.com:80
  - HTTPS_PROXY=http://proxy.example.com:80
  - NO_PROXY=localhost,127.0.0.1

ssh_config:
  user: ubuntu
  identity_file: ~/.ssh/id_rsa
  hosts:
    - 3.255.177.51
    - 3.255.177.52

$ dstack fleet

 FLEET     INSTANCE  BACKEND              GPU             PRICE    STATUS  CREATED 
 my-fleet  0         gcp (europe-west-1)  L4:24GB (spot)  \$0.1624  idle    3 mins ago      
           1         gcp (europe-west-1)  L4:24GB (spot)  \$0.1624  idle    3 mins ago    

$ dstack delete -f cluster.dstack.yaml
Delete the fleet my-gcp-fleet? [y/n]: y
Fleet my-gcp-fleet deleted

$ dstack offer --gpu H100 --fleet my-fleet