> ## Documentation Index
> Fetch the complete documentation index at: https://training-docs.cerebras.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Evaluation Metrics 

> Learn to use and create metrics in Cerebras for evaluating PyTorch models, including predefined metrics like AccuracyMetric and custom metrics tailored to specific evaluation needs.

We provide Cerebras-compatible metrics that can be used during evaluation to measure how well the model has been trained. These metrics can be found in the [metrics.Metric](https://training-api.cerebras.ai/en/latest/wsc/api/cerebras_pytorch/metrics.html#metrics) module.

For example:

```python theme={null}
model = Model()
compiled_model = cstorch.compile(model, backend)

accuracy = cstorch.metrics.AccuracyMetric("accuracy")

@cstorch.trace
def eval_step(batch):
    inputs, targets = batch
    outputs = compiled_model(inputs)

    accuracy(
        labels=targets.clone(),
        predictions=outputs.argmax(-1).int(),
    )

...
for batch in executor:
    ...


# Log accumulated eval metric
print(f"Accuracy: {float(accuracy)}")
```

## Writing Custom Metrics

To define a Cerebras compliant metrics, create a subclass of [`cerebras.pytorch.metrics.Metric`](https://training-api.cerebras.ai/en/latest/wsc/api/cerebras_pytorch/metrics.html#cerebras.pytorch.metrics.Metric "cerebras.pytorch.metrics.Metric").

For example,

```python theme={null}
class CustomMetric(cstorch.metrics.Metric):

    def __init__(self, name: str):
        super().__init__(name)
        ...

    ...

    def reset(self):
        pass

    def update(self, *args, **kwargs):
        ...

    def compute(self):
        ...
```

As can be seen in the above example, the base [`Metric`](https://training-api.cerebras.ai/en/latest/wsc/api/cerebras_pytorch/metrics.html#cerebras.pytorch.metrics.Metric "cerebras.pytorch.metrics.Metric") class expects one argument. Namely, the metric name.

In addition, there are three abstract methods that must be overridden:

* [`reset`](https://training-api.cerebras.ai/en/latest/wsc/api/cerebras_pytorch/metrics.html#cerebras.pytorch.metrics.Metric.reset "cerebras.pytorch.metrics.Metric.reset")
  This method resets (or defines if its the first time its called) the metrics’ internal state.States can be registered via calls to [`register_state`](https://training-api.cerebras.ai/en/latest/wsc/api/cerebras_pytorch/metrics.html#cerebras.pytorch.metrics.Metric.register_state "cerebras.pytorch.metrics.Metric.register_state")

* [`update`](https://training-api.cerebras.ai/en/latest/wsc/api/cerebras_pytorch/metrics.html#cerebras.pytorch.metrics.Metric.update "cerebras.pytorch.metrics.Metric.update")
  This method is used to update the metric’s registered states.Note that to remain Cerebras compliant, no tensor may be evaluated/inspected here. The update call is intended to be fully traced.

* [`compute`](https://training-api.cerebras.ai/en/latest/wsc/api/cerebras_pytorch/metrics.html#cerebras.pytorch.metrics.Metric.compute "cerebras.pytorch.metrics.Metric.compute")
  This method is used to compute the final accumulated metric value using the state that was updated in [`update`](https://training-api.cerebras.ai/en/latest/wsc/api/cerebras_pytorch/metrics.html#cerebras.pytorch.metrics.Metric.update)