> ## Documentation Index
> Fetch the complete documentation index at: https://training-docs.cerebras.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Numeric Precision

One thing you can do to enhance performance is to play around with the numeric precision level. Using lower precisions can drastically improve performance at the cost of numeric precision. For optimal performance and numeric stability, it is essential to utilize all the recommended methods.

This page will cover how you can configure numeric precision using the [`Trainer`](https://training-api.cerebras.ai/en/latest/wsc/Model-zoo/api/index.html) and ways you can mitigate some of the adverse effects of using lower precision.

## Prerequisites

Please ensure that you have read through the [Trainer Overview](../../../model-zoo/trainer-overview) beforehand. The rest of this page assumes that you already have at least a cursory understanding of what the Cerebras Model Zoo Trainer is and how to use the python API.

Also, read through the [Gradient scaling](../../../cs-torch/writing-a-custom-training-loop/gradient-scaling) guide for more context on the concepts being used below.

## Automatic Mixed Precision

Using a lower precision for computating activations while storing weights in a higher precision is a good way to get improved performance whilst keeping some numeric stability.

To facilitate this, you can construct a [`MixedPrecision`](https://docs.cerebras.net/en/latest/wsc/Model-zoo/api/generated/cerebras.modelzoo.trainer.callbacks.html#cerebras.modelzoo.trainer.callbacks.MixedPrecision"cerebras.modelzoo.trainer.callbacks.MixedPrecision") instance and pass it into the [`Trainer`](https://training-api.cerebras.ai/en/latest/wsc/Model-zoo/api/index.html)’s `precision` argument as follows.

<CodeGroup>
  ```yaml YAML theme={null}

  trainer:
    init:
      ...
      precision:
        fp16_type: cbfloat16

  ```

  ```python Python theme={null}
  from cerebras.modelzoo import Trainer
  from cerebras.modelzoo.trainer.callbacks import MixedPrecision

  trainer = Trainer(
      ...,
      precision=MixedPrecision(
          fp16_type="cbfloat16",
      ),
  )
  ```
</CodeGroup>

In the above example, the lower precision type is set to `cbfloat16`. The supported lower precision values include:

* `float16`

* `bfloat16`

* `cbfloat16`

## Gradient Scaling

When using a lower numeric precision, you will often encounter gradient underflow. To mitigate this, you can employ gradient scaling (see [Gradient scaling](../../../cs-torch/writing-a-custom-training-loop/gradient-scaling) for a more in-depth explanation).

To configure gradient scaling, you can pass in the `loss_scaling_factor` argument to [`MixedPrecision`](https://https://training-api.cerebras.ai/en/latest/wsc/Model-zoo/api/generated/cerebras.modelzoo.trainer.callbacks.html#cerebras.modelzoo.trainer.callbacks.MixedPrecision"cerebras.modelzoo.trainer.callbacks.MixedPrecision") as follows:

<CodeGroup>
  ```yaml YAML theme={null}

  trainer:
    init:
      ...
      precision:
        fp16_type: cbfloat16
        loss_scaling_factor: dynamic

  ```

  ```python Python theme={null}
  from cerebras.modelzoo import Trainer
  from cerebras.modelzoo.trainer.callbacks import MixedPrecision

  trainer = Trainer(
      ...,
      precision=MixedPrecision(
          fp16_type="cbfloat16",
          loss_scaling_factor="dynamic",
      ),
  )

  ```
</CodeGroup>

`loss_scaling_factor` accepts either some `float` for static loss scaling, or the string `"dynamic"` to facilitate dynamic loss scaling (see [Dynamic loss scaling](../../../model-zoo/tutorials/dynamic-loss-scaling) for more details).

## Gradient Clipping

Even with all of the above, you may encounter exploding gradients (`inf` or `NaN` gradients). To mitigate this, you can employ the use of gradient clipping.

To configure gradient clipping, you can pass in one of `max_gradient_norm` or `max_gradient_value` to [`MixedPrecision`](https://https://training-api.cerebras.ai/en/latest/wsc/Model-zoo/api/generated/cerebras.modelzoo.trainer.callbacks.html#cerebras.modelzoo.trainer.callbacks.MixedPrecision"cerebras.modelzoo.trainer.callbacks.MixedPrecision") as follows:

<CodeGroup>
  ```yaml YAML theme={null}

  trainer:
    init:
      ...
      precision:
        fp16_type: cbfloat16
        loss_scaling_factor: dynamic
        max_gradient_norm: 1.0
        # OR
        max_gradient_value: 2.0

  ```

  ```python Python theme={null}

  from cerebras.modelzoo import Trainer
  from cerebras.modelzoo.trainer.callbacks import MixedPrecision

  trainer = Trainer(
      ...,
      precision=MixedPrecision(
          fp16_type="cbfloat16",
          loss_scaling_factor="dynamic",
          max_gradient_norm=1.0,
          # OR
          max_gradient_value=2.0,
      ),
  )
  ```
</CodeGroup>

These two arguments correspond directly to calls to

* [torch.nn.utils.clip\_grad\_norm\_](https://pytorch.org/docs/stable/generated/torch.nn.utils.clip_grad_norm_.html)

* [torch.nn.utils.clip\_grad\_value\_](https://pytorch.org/docs/stable/generated/torch.nn.utils.clip_grad_value_.html)

<Note>
  `max_gradient_norm` and `max_gradient_value` are mutually exclusive. So, only one may be passed in.
</Note>

## Precision Optimization Level (POL)

One additional setting you can configure to improve performance is to set the precision optimization level used by the Cerebras compiler.

The precision optimization level (POL) to use when compiling the model. The POL determines the level of precision to use for the model’s weights and activations and can thus affect the model’s accuracy and performance.

You can set the precision optimization level by passing in the `precision_opt_level` argument to [`MixedPrecision`](https://training-api.cerebras.ai/en/latest/wsc/Model-zoo/api/generated/cerebras.modelzoo.trainer.callbacks.html#cerebras.modelzoo.trainer.callbacks.MixedPrecision"cerebras.modelzoo.trainer.callbacks.MixedPrecision") as follows:

<CodeGroup>
  ```yaml YAML theme={null}

  trainer:
    init:
      ...
      precision:
        fp16_type: cbfloat16
        loss_scaling_factor: dynamic
        max_gradient_norm: 1.0
        precision_opt_level: 1

  ```

  ```python Python theme={null}
  from cerebras.modelzoo import Trainer
  from cerebras.modelzoo.trainer.callbacks import MixedPrecision

  trainer = Trainer(
      ...,
      precision=MixedPrecision(
          fp16_type="float16",
          loss_scaling_factor="dynamic",
          max_gradient_norm=1.0,
          precision_opt_level=1,
      ),
  )

  ```
</CodeGroup>

The value must be one of `[0, 1, 2]`. The precision optimization level is set to `1` by default.

## Conclusion

That is all there is to know about configuring numeric precision in the [`Trainer`](https://training-api.cerebras.ai/en/latest/wsc/Model-zoo/api/index.html)!

### Further Reading

To learn more about how you can extend the capabilities of the [`Trainer`](https://training-api.cerebras.ai/en/latest/wsc/Model-zoo/api/index.html) class, you can check out:

* [Defer Weight Initialization](../../../model-zoo/components/trainer-components/defer-weight-initialization)

* [Train a model with weight sparsity](../../../model-zoo/tutorials/train-a-model-with-weight-sparsity)

* [Checkpointing](../../../model-zoo/components/trainer-components/checkpointing)

* [Customizing the Trainer with Callbacks](../../../model-zoo/components/trainer-components/customizing-the-trainer-with-callbacks)

* [Logging](../../../model-zoo/components/trainer-components/logging)