> ## Documentation Index
> Fetch the complete documentation index at: https://training-docs.cerebras.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Supported Config Parameters

> Learn about supported config parameters within Model Zoo.

The Cerebras Model Zoo supports an extensive range of models, each with its own set of parameters. This document provides a comprehensive list of these parameters as defined in the Model Zoo. With the introduction of Config classes in the Model Zoo, each parameter is defined in the model’s `config file`. These classes, implemented as Python dataclasses, organize and validate the parameters necessary for model definition and training.

Common parameters across models include:

1. RunConfig Parameters:

[`RunConfig`](https://docs.cerebras.net/en/latest/wsc/port/_autosummary/cerebras.modelzoo.config_manager.config_classes.base.run_config.RunConfig.html#cerebras.modelzoo.config_manager.config_classes.base.run_config.RunConfig "cerebras.modelzoo.config_manager.config_classes.base.run_config.RunConfig")

1. Sparsity Parameters:

[`SparsityBaseConfig`](https://docs.cerebras.net/en/latest/wsc/port/_autosummary/cerebras.modelzoo.config_manager.config_classes.base.sparsity_config.SparsityBaseConfig.html#cerebras.modelzoo.config_manager.config_classes.base.sparsity_config.SparsityBaseConfig "cerebras.modelzoo.config_manager.config_classes.base.sparsity_config.SparsityBaseConfig")

1. Optimizer Parameters:

[`OptimizerConfig`](https://docs.cerebras.net/en/latest/wsc/port/_autosummary/cerebras.modelzoo.config_manager.config_classes.base.optimizer_config.OptimizerConfig.html#cerebras.modelzoo.config_manager.config_classes.base.optimizer_config.OptimizerConfig "cerebras.modelzoo.config_manager.config_classes.base.optimizer_config.OptimizerConfig")

## Model Specific Parameters

<AccordionGroup>
  <Accordion title="Large Language Model (LLM) Parameters" defaultOpen={false}>
    * [BERT](https://github.com/Cerebras/modelzoo/tree/main/src/cerebras/modelzoo//models/nlp/bert/config.py)

    * [Bloom ](https://github.com/Cerebras/modelzoo/tree/main/src/cerebras/modelzoo//models/nlp/bloom/config.py)

    * [BTLM config](https://github.com/Cerebras/modelzoo/tree/main/src/cerebras/modelzoo//models/nlp/btml/config.py)

    * [DPO config](https://github.com/Cerebras/modelzoo/tree/main/src/cerebras/modelzoo//models/nlp/dpo/config.py)

    * [DPR](https://github.com/Cerebras/modelzoo/tree/main/src/cerebras/modelzoo//models/nlp/dpr/config.py)

    * [Falcon](https://github.com/Cerebras/modelzoo/tree/main/src/cerebras/modelzoo//models/nlp/falcon/config.py)

    * [GPT2](https://github.com/Cerebras/modelzoo/tree/main/src/cerebras/modelzoo//models/nlp/gpt2/config.py)

    * [GPT3](https://github.com/Cerebras/modelzoo/tree/main/src/cerebras/modelzoo//models/nlp/gpt3/config.py)

    * [GPTJ](https://github.com/Cerebras/modelzoo/tree/main/src/cerebras/modelzoo//models/nlp/gptj/config.py)

    * [Llama](https://github.com/Cerebras/modelzoo/tree/main/src/cerebras/modelzoo//models/nlp/llama/config.py)

    * [Mistral](https://github.com/Cerebras/modelzoo/tree/main/src/cerebras/modelzoo//models/nlp/mistral/config.py)

    * [SantaCoder](https://github.com/Cerebras/modelzoo/tree/main/src/cerebras/modelzoo//models/nlp/santacoder/config.py)

    * [StarCoder](https://github.com/Cerebras/modelzoo/tree/main/src/cerebras/modelzoo//models/nlp/santacoder/config.py)

    * [T5](https://github.com/Cerebras/modelzoo/tree/main/src/cerebras/modelzoo//models/nlp/t5/config.py)

    * [Transformer ](https://github.com/Cerebras/modelzoo/tree/main/src/cerebras/modelzoo//models/nlp/transformer/config.py)
  </Accordion>

  <Accordion title="Vision Model Parameters" defaultOpen={false}>
    * [DiT config](https://github.com/Cerebras/modelzoo/tree/main/src/cerebras/modelzoo//models/vision/dit/config.py)

    * [Vision Transformer config](https://github.com/Cerebras/modelzoo/tree/main/src/cerebras/modelzoo//models/vision/vision_transformer/config.py)
  </Accordion>

  <Accordion title="Multimodal Model Parameters" defaultOpen={false}>
    [LLaVA config](https://github.com/Cerebras/modelzoo/tree/main/src/cerebras/modelzoo//models/multimodal/llava/config.py)
  </Accordion>
</AccordionGroup>

## Understanding Config Class Structure

Each of these parameters are present as part of a Config class. A Config class, implemented as a Python dataclass, serves as a container for storing essential settings and parameters needed for defining and training a model.

Each of these class atributes correspond to the respective section in a YAML file which is used to define the parameters for a training run.

A config class looks like this:

```
class <ConfigClass>:
    train_input = Optional[DataConfig] = None

    eval_input = Optional[DataConfig] = None

    model = <ModelConfigClass> = required

    sparsity: Optional[SparsityConfig] = None

    optimizer: OptimizerConfig = required

    runconfig: RunConfig = required

```

For more information about Config classes, refer to the [Model Zoo config classes](../../model-zoo/components/model-zoo-config-classes) documentation.

## Additional notes

Each model has a designated `ModelConfigClass`. In cases where a model is a variant of another, it may inherit the `ModelConfigClass` from the parent model. To understand more about this inheritance and the hierarchy of config classes, visit the [Config class hierarchy](../../model-zoo/components/model-zoo-config-classes#config-class-hierarchy) documentation.