> ## Documentation Index
> Fetch the complete documentation index at: https://training-docs.cerebras.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Backend

> Learn how to set up a backend or device for the Trainer class.

The [`cerebras.pytorch.backend`](https://training-api.cerebras.ai/en/latest/wsc/api/cerebras_pytorch/cstorch.html#backend) is the configuration of the device and other settings used during a run. The `device` is what hardware the workflow will run on.

## Prerequisites

Make sure to have read through [Trainer Overview](../../../model-zoo/trainer-overview) and [Trainer Configuration Overview](../../../model-zoo/trainer-configuration-overview) which provide the basic overview of how to run Model Zoo models. In this document, you will be using the tools and configurations outlined in those pages.

## Configure the Device

Configuring the device used by the [`Trainer`](https://training-api.cerebras.ai/en/latest/wsc/Model-zoo/api/index.html) can be done by simply specifying one of `"CSX"`, `"CPU"`, or `"GPU"`.

<CodeGroup>
  ```yaml YAML theme={null}

  trainer:
    init:
      device: "CSX"
      ...
    ...

  ```

  ```python Python theme={null}
  from cerebras.modelzoo import Trainer

  trainer = Trainer(
      device="CSX",
      ...,
  )
  ...

  ```
</CodeGroup>

<Note>
  Setting `device` still creates a `cerebras.pytorch.backend` instance just with default settings. To configure anything about the backend, you must specify those parameters via the `backend` key instead.
</Note>

### Limitations

Once a device is set, any other `Trainer` instances must also use the same device type as well. You cannot mix device types. For example, a configuration like this:

<CodeGroup>
  ```yaml YAML theme={null}

  # THIS CONFIGURATION IS INVALID
  trainer:
  - trainer:
      init:
        device: "CSX"
        ...
      ...
  - trainer:
      init:
        device: "CPU"
        ...
      ...

  ```

  ```python Python theme={null}
  from cerebras.modelzoo import Trainer

  # THIS CONFIGURATION IS INVALID
  trainer1 = Trainer(
      device="CSX",
      ...,
  )

  trainer2 = Trainer(
      device="CPU",
      ...,
  )
  ...

  ```
</CodeGroup>

will result in the following error:

```bash theme={null}
RuntimeError: Cannot instantiate multiple backends. A backend with type CSX has already been instantiated.

```

## Configure the Backend

Configuring the backend used by the `Trainer` can be done by creating a `cerebras.pytorch.backend` instance.

The configuration is expected to be a dictionary whose keys will be used to construct a `cerebras.pytorch.backend` instance.

In the Python script, construct a `cerebras.pytorch.backend` instance and pass it to the `backend` argument.

<CodeGroup>
  ```yaml YAML theme={null}
  trainer:
    init:
      backend:
        backend_type: "CSX"
        cluster_config:
          num_csx: 4
          mount_dirs:
          - /path/to/dir1
          - /path/to/dir2
          ...
        ...
      ...
    ...

  ```

  ```python Python theme={null}
  import cerebras.pytorch as cstorch
  from cerebras.modelzoo import Trainer

  trainer = Trainer(
      backend=cstorch.backend(
          backend_type="CSX",
          cluster_config=cstorch.distributed.ClusterConfig(
              num_csx=4,
              mount_dirs=["/path/to/dir1", "/path/to/dir2"],
              ...,
          ),
          ...,
      )
      ...
  )
  ...

  ```
</CodeGroup>

### Limitations

Multiple backend instantiations with different devices is not supported. You will see this error:

```bash theme={null}
RuntimeError: Cannot instantiate multiple backends. A backend with type CSX has already been instantiated.

```

That means that when you construct one or more `Trainer` instances, you must ensure you only instantiate backends of a single device type. However you can change other backend parameters between `Trainer` instances.

The configuration is expected to be a dictionary whose keys will be used to construct a `cerebras.pytorch.backend` instance.

In the Python script, construct a `cerebras.pytorch.backend` instance and pass it to the `backend` argument.

For example:

<CodeGroup>
  ```yaml YAML theme={null}
  trainer:
  - trainer:
      init:
        backend:
          backend_type: "CSX"
          cluster_config:
            num_csx: 4
            mount_dirs:
            - /path/to/dir1
            - /path/to/dir2
            ...
          ...
        ...
      ...
  - trainer:
      init:
        backend:
          backend_type: "CSX"
          cluster_config:
            num_csx: 2
            num_workers_per_csx: 1
            mount_dirs:
            - /path/to/dir1
            - /path/to/dir2
            ...
          ...
        ...
      ...

  ```

  ```python Python theme={null}
  import cerebras.pytorch as cstorch
  from cerebras.modelzoo import Trainer

  backend = cstorch.backend(
      "CSX",
      cluster_config=cstorch.distributed.ClusterConfig(
          num_csx=4,
          mount_dirs=["/path/to/dir1", "/path/to/dir2"],
          ...,
      ),
  )

  trainer1 = Trainer(
      backend=backend,
      ...
  )

  backend.cluster_config.num_csx = 2
  backend.cluster_config.num_workers_per_csx = 1

  trainer2 = Trainer(
      backend=backend,
      ...
  )
  ```
</CodeGroup>

## Mutual Exclusivity

The `device` and `backend` arguments are mutually exclusive. It is expected when initializing a `Trainer` to set one of them but not both. If both are set, you will see an error that looks like this:

```bash theme={null}
ValueError: backend and device are mutually exclusive arguments of Trainer. Please only provide one or the other

```

## Further Reading

To learn more about how you can use the [`Trainer`](https://training-api.cerebras.ai/en/latest/wsc/Model-zoo/api/index.html) in some core workflows, you can check out:

* [Pretraining with Upstream Validation](../../../model-zoo/core-workflows/pretraining-with-upstream-validation)

To learn more about how you can extend the capabilities of the `Trainer` class, you can check out:

* [Defer Weight Initialization](../../../model-zoo/components/trainer-components/defer-weight-initialization)

* [Numeric Precision](../../../model-zoo/components/trainer-components/numeric-precision)

* [Train a model with weight sparsity](../../../model-zoo/tutorials/train-a-model-with-weight-sparsity)

* [Checkpointing](../../../model-zoo/components/trainer-components/checkpointing)

* [Customizing the Trainer with Callbacks](../../../model-zoo/components/trainer-components/customizing-the-trainer-with-callbacks)

* [Logging](../../../model-zoo/components/trainer-components/logging)