The Cerebras Wafer-Scale Cluster is meticulously engineered to enable the training of neural networks with remarkably efficient linear scaling across millions of cores, all without the complexities of traditional distributed computing.
Fig. 1: Topology of Cerebras Wafer-Scale Cluster
Key Components
CS-3 Systems
The CS-3 systems represents the latest advancement in our lineup, incorporating the innovative WSE-3 (Wafer-Scale Engine 3) to deliver cutting-edge performance. Building on the user-friendly experience of its predecessor, the CS-2, the CS-3 significantly enhances computational efficiency and performance.This next-gen system introduces a novel instruction set architecture and boosts the SIMD (Single Instruction, Multiple Data) width, providing more parallel processing power. Additionally, it expands the core count by 20% and enhances the ingress/egress bandwidth by 30%, enabling faster data processing and transfer rates.Compatibility and performance testing are a cornerstone of the CS-3’s development. All models available in the Model Zoo have been verified for functionality on the CS-3 platform. Furthermore, a selection of key models, including GPT-2, GPT-3, LLaMA, Falcon, BTLM, GPT-J, GPT-NeoX, T5, BERT, and PaLI have undergone rigorous testing to ensure both convergence and performance are optimized for the CS-3, guaranteeing a seamless and efficient user experience. You can find more detailed information in the WSE-3 datasheet and explore CS-3.
MemoryX Technology
MemoryX technology serves as the storage and intelligent streaming solution for a model’s weights, ensuring efficient and timely access for the CS-X systems.
SwarmX Technology
SwarmX technology plays a pivotal role in integrating multiple CS-X nodes into a unified Cerebras cluster. These nodes collaborate seamlessly in training a single model. SwarmX handles the broadcast of weights from MemoryX to the entire cluster and effectively reduces (sums) gradients in the opposite direction, contributing to efficient training processes.
Input Preprocessing Servers
Input Preprocessing servers handle the critical task of preprocessing training data, ensuring that it is appropriately prepared before being dispatched to the CS-X systems. This preprocessing step is vital for training, inference, and evaluation.
Management Servers
Management servers are responsible for orchestrating and scheduling the cluster’s resources, ensuring efficient utilization and coordination among all components in the Cerebras cluster. They play a key role in optimizing the cluster’s performance and resource allocation.
Working with the Cerebras Cluster
After developing your code, you can initiate the process of submitting it for both training and evaluation from a user node. It’s important to note that the user node operates independently from the cluster and connects to the Cerebras cluster through the management server, as illustrated in Fig. 1 above. The management server is responsible for handling all resource scheduling. Therefore, your main task is to specify the number of CS-X systems you wish to allocate for either training or evaluation. This enables you to efficiently utilize the Cerebras cluster’s resources without directly managing the intricacies of cluster allocation.For documentation related to the installation and administration of the Cerebras Wafer-Scale cluster, visit Cerebras deployment documentation.
⌘I
{"docsConfig":{"theme":"almond","$schema":"https://mintlify.com/docs.json","name":"Cerebras Training","colors":{"primary":"#F1592A","light":"#FFFFFF","dark":"#0C0C0E"},"logo":{"light":"https://mintcdn.com/cerebras-training/hh6MqoQlI8YKsSkL/logo/2.svg?fit=max\u0026auto=format\u0026n=hh6MqoQlI8YKsSkL\u0026q=85\u0026s=9430343b4d9c7548560af71291450f1f","dark":"https://mintcdn.com/cerebras-training/hh6MqoQlI8YKsSkL/logo/1.svg?fit=max\u0026auto=format\u0026n=hh6MqoQlI8YKsSkL\u0026q=85\u0026s=5e459bac245b767748a20ecad9b86991"},"favicon":"https://mintcdn.com/cerebras-training/svm74p9sumHaelW6/favicon.png?fit=max\u0026auto=format\u0026n=svm74p9sumHaelW6\u0026q=85\u0026s=9fb0ab4fc01d95d9b772e28f11586850","background":{"decoration":"gradient"},"navbar":{"primary":{"type":"button","label":"Contact Us","href":"https://www.cerebras.net/contact-us/"}},"footer":{"socials":{"discord":"https://discord.com/invite/ZqvYS2e2rY","github":"https://github.com/Cerebras/modelzoo","linkedin":"https://www.linkedin.com/company/cerebras-systems","youtube":"https://www.youtube.com/channel/UCAAJD_MScghZj9R1cUZ3c8w"}},"fonts":{"heading":{"family":"Manrope"},"body":{"family":"Manrope"}},"styling":{"eyebrows":"breadcrumbs","latex":true},"integrations":{"gtm":{"tagId":"GTM-KBZT7S7N"}},"contextual":{"options":["copy","mcp","chatgpt","claude","perplexity","cursor","vscode"],"display":"header"},"navigation":{"global":{"anchors":[{"anchor":"Community","icon":"discord","href":"https://discord.com/invite/ZqvYS2e2rY"},{"anchor":"API Reference","icon":"code","href":"https://training-api.cerebras.ai"}]},"pages":[]}},"deployment":{"deploymentMetadataValue":{"subdomain":"cerebras-training","hasAudioNarration":false,"actualSubdomain":"cerebras-training","customDomains":["training-docs.cerebras.ai"],"gitSource":{"type":"github","owner":"cerebras","repo":"training-docs","deployBranch":"main","contentDirectory":"","isPrivate":true,"mountPath":"","label":""},"entitlements":{"AI_CHAT":{"status":"ENABLED"},"REMOVE_BRANDING":{"status":"ENABLED"},"CONTEXTUAL_FEEDBACK":{"status":"ENABLED"},"CODE_SNIPPET_FEEDBACK":{"status":"ENABLED"},"WEB_MCP":{"status":"ENABLED"},"NARRATION":{"status":"ENABLED"}},"buildId":"6afdda8f8443deadda263085","clientVersion":"0.0.4248","searchFilterCounts":[{"language":"default","tag":"Guides","count":505},{"language":"default","tag":"Explore \u0026 Learn","count":348},{"language":"default","tag":"Models","count":126},{"language":"default","tag":"Training and Eval","count":117},{"language":"default","tag":"Concepts \u0026 Guides","count":112},{"language":"default","tag":"Language Models","count":96},{"language":"default","tag":"Tutorials","count":96},{"language":"default","tag":"Writing a Custom Training Loop","count":91},{"language":"default","tag":"Trainer Components","count":80},{"language":"default","tag":"Data Preparation","count":78}]},"assistantConfig":{"deflection":{"enabled":false,"showHelpButton":false}},"basePath":"","hasChatPermissions":true,"hasWebMcpPermissions":true,"toggles":[{"name":"kill-isr","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"custom-rbac-roles","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"agent-mintlify-slack","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"agent-integrations-settings","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"rbac-expansion","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"automations-pr-grouping","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"custom-prefix","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"analytics-v7","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"user-flows-analytics","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"no-publish-flow","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"workos-onboarding","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"snapshot-serving-live","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"stripTemperatureMiddleware","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"support-platform-pylon-zendesk","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"snapshot-serving","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"instant-rollback","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"granular-notifications-server","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"granular-notifications","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"automations-fable","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"fixed-ai-action-pricing","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"site-roles","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"automation-resolve-merge-conflict","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"scoped-publishing","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"mintql","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"page-level-analytics","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"editor-version-history-v2","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"billing-page-v2","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"full-page-search","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"workspace-projects","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"assistant-v5","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"fork-pr-previews","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"revalidation-jwt","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"workos-sso-scim","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"hobby-abuse-scan","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"dashboard-inbox","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"workos-signup","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"turbopuffer-dual-write","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"workos-enabled","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"approve-review-flow","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"editor-metadata","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"sso-setup-deadline-banner","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"turbopuffer-serve","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"live-agents","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"starter-pricing","enabled":true,"variant":{"name":"control","enabled":true,"payload":{"type":"string","value":"control"},"feature_enabled":true},"impressionData":false},{"name":"llms-txt-v2","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"wildcard-email-auth","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false}],"googleFontsInlined":true},"customCss":[{"filePath":"style.css","content":"/* cyrillic-ext */\n@font-face {\n font-family: 'Manrope';\n font-style: normal;\n font-weight: 300;\n font-display: swap;\n src: url(/_mintlify/assets/fonts/google/s/manrope/v23/xn7gYHE41ni1AdIRggqxSuXd.woff2) format('woff2');\n unicode-range: U+0460-052F, U+1C80-1C8A, U+20B4, U+2DE0-2DFF, U+A640-A69F, U+FE2E-FE2F;\n}\n/* cyrillic */\n@font-face {\n font-family: 'Manrope';\n font-style: normal;\n font-weight: 300;\n font-display: swap;\n src: url(/_mintlify/assets/fonts/google/s/manrope/v23/xn7gYHE41ni1AdIRggOxSuXd.woff2) format('woff2');\n unicode-range: U+0301, U+0400-045F, U+0490-0491, U+04B0-04B1, U+2116;\n}\n/* greek */\n@font-face {\n font-family: 'Manrope';\n font-style: normal;\n font-weight: 300;\n font-display: swap;\n src: url(/_mintlify/assets/fonts/google/s/manrope/v23/xn7gYHE41ni1AdIRggSxSuXd.woff2) format('woff2');\n unicode-range: U+0370-0377, U+037A-037F, U+0384-038A, U+038C, U+038E-03A1, U+03A3-03FF;\n}\n/* vietnamese */\n@font-face {\n font-family: 'Manrope';\n font-style: normal;\n font-weight: 300;\n font-display: swap;\n src: url(/_mintlify/assets/fonts/google/s/manrope/v23/xn7gYHE41ni1AdIRggixSuXd.woff2) format('woff2');\n unicode-range: U+0102-0103, U+0110-0111, U+0128-0129, U+0168-0169, U+01A0-01A1, U+01AF-01B0, U+0300-0301, U+0303-0304, U+0308-0309, U+0323, U+0329, U+1EA0-1EF9, U+20AB;\n}\n/* latin-ext */\n@font-face {\n font-family: 'Manrope';\n font-style: normal;\n font-weight: 300;\n font-display: swap;\n src: url(/_mintlify/assets/fonts/google/s/manrope/v23/xn7gYHE41ni1AdIRggmxSuXd.woff2) format('woff2');\n unicode-range: U+0100-02BA, U+02BD-02C5, U+02C7-02CC, U+02CE-02D7, U+02DD-02FF, U+0304, U+0308, U+0329, U+1D00-1DBF, U+1E00-1E9F, U+1EF2-1EFF, U+2020, U+20A0-20AB, U+20AD-20C4, U+2113, U+2C60-2C7F, U+A720-A7FF;\n}\n/* latin */\n@font-face {\n font-family: 'Manrope';\n font-style: normal;\n font-weight: 300;\n font-display: swap;\n src: url(/_mintlify/assets/fonts/google/s/manrope/v23/xn7gYHE41ni1AdIRggexSg.woff2) format('woff2');\n unicode-range: U+0000-00FF, U+0131, U+0152-0153, U+02BB-02BC, U+02C6, U+02DA, U+02DC, U+0304, U+0308, U+0329, U+2000-206F, U+20AC, U+2122, U+2191, U+2193, U+2212, U+2215, U+FEFF, U+FFFD;\n}\n/* cyrillic-ext */\n@font-face {\n font-family: 'Manrope';\n font-style: normal;\n font-weight: 400;\n font-display: swap;\n src: url(/_mintlify/assets/fonts/google/s/manrope/v23/xn7gYHE41ni1AdIRggqxSuXd.woff2) format('woff2');\n unicode-range: U+0460-052F, U+1C80-1C8A, U+20B4, U+2DE0-2DFF, U+A640-A69F, U+FE2E-FE2F;\n}\n/* cyrillic */\n@font-face {\n font-family: 'Manrope';\n font-style: normal;\n font-weight: 400;\n font-display: swap;\n src: url(/_mintlify/assets/fonts/google/s/manrope/v23/xn7gYHE41ni1AdIRggOxSuXd.woff2) format('woff2');\n unicode-range: U+0301, U+0400-045F, U+0490-0491, U+04B0-04B1, U+2116;\n}\n/* greek */\n@font-face {\n font-family: 'Manrope';\n font-style: normal;\n font-weight: 400;\n font-display: swap;\n src: url(/_mintlify/assets/fonts/google/s/manrope/v23/xn7gYHE41ni1AdIRggSxSuXd.woff2) format('woff2');\n unicode-range: U+0370-0377, U+037A-037F, U+0384-038A, U+038C, U+038E-03A1, U+03A3-03FF;\n}\n/* vietnamese */\n@font-face {\n font-family: 'Manrope';\n font-style: normal;\n font-weight: 400;\n font-display: swap;\n src: url(/_mintlify/assets/fonts/google/s/manrope/v23/xn7gYHE41ni1AdIRggixSuXd.woff2) format('woff2');\n unicode-range: U+0102-0103, U+0110-0111, U+0128-0129, U+0168-0169, U+01A0-01A1, U+01AF-01B0, U+0300-0301, U+0303-0304, U+0308-0309, U+0323, U+0329, U+1EA0-1EF9, U+20AB;\n}\n/* latin-ext */\n@font-face {\n font-family: 'Manrope';\n font-style: normal;\n font-weight: 400;\n font-display: swap;\n src: url(/_mintlify/assets/fonts/google/s/manrope/v23/xn7gYHE41ni1AdIRggmxSuXd.woff2) format('woff2');\n unicode-range: U+0100-02BA, U+02BD-02C5, U+02C7-02CC, U+02CE-02D7, U+02DD-02FF, U+0304, U+0308, U+0329, U+1D00-1DBF, U+1E00-1E9F, U+1EF2-1EFF, U+2020, U+20A0-20AB, U+20AD-20C4, U+2113, U+2C60-2C7F, U+A720-A7FF;\n}\n/* latin */\n@font-face {\n font-family: 'Manrope';\n font-style: normal;\n font-weight: 400;\n font-display: swap;\n src: url(/_mintlify/assets/fonts/google/s/manrope/v23/xn7gYHE41ni1AdIRggexSg.woff2) format('woff2');\n unicode-range: U+0000-00FF, U+0131, U+0152-0153, U+02BB-02BC, U+02C6, U+02DA, U+02DC, U+0304, U+0308, U+0329, U+2000-206F, U+20AC, U+2122, U+2191, U+2193, U+2212, U+2215, U+FEFF, U+FFFD;\n}\n/* cyrillic-ext */\n@font-face {\n font-family: 'Manrope';\n font-style: normal;\n font-weight: 500;\n font-display: swap;\n src: url(/_mintlify/assets/fonts/google/s/manrope/v23/xn7gYHE41ni1AdIRggqxSuXd.woff2) format('woff2');\n unicode-range: U+0460-052F, U+1C80-1C8A, U+20B4, U+2DE0-2DFF, U+A640-A69F, U+FE2E-FE2F;\n}\n/* cyrillic */\n@font-face {\n font-family: 'Manrope';\n font-style: normal;\n font-weight: 500;\n font-display: swap;\n src: url(/_mintlify/assets/fonts/google/s/manrope/v23/xn7gYHE41ni1AdIRggOxSuXd.woff2) format('woff2');\n unicode-range: U+0301, U+0400-045F, U+0490-0491, U+04B0-04B1, U+2116;\n}\n/* greek */\n@font-face {\n font-family: 'Manrope';\n font-style: normal;\n font-weight: 500;\n font-display: swap;\n src: url(/_mintlify/assets/fonts/google/s/manrope/v23/xn7gYHE41ni1AdIRggSxSuXd.woff2) format('woff2');\n unicode-range: U+0370-0377, U+037A-037F, U+0384-038A, U+038C, U+038E-03A1, U+03A3-03FF;\n}\n/* vietnamese */\n@font-face {\n font-family: 'Manrope';\n font-style: normal;\n font-weight: 500;\n font-display: swap;\n src: url(/_mintlify/assets/fonts/google/s/manrope/v23/xn7gYHE41ni1AdIRggixSuXd.woff2) format('woff2');\n unicode-range: U+0102-0103, U+0110-0111, U+0128-0129, U+0168-0169, U+01A0-01A1, U+01AF-01B0, U+0300-0301, U+0303-0304, U+0308-0309, U+0323, U+0329, U+1EA0-1EF9, U+20AB;\n}\n/* latin-ext */\n@font-face {\n font-family: 'Manrope';\n font-style: normal;\n font-weight: 500;\n font-display: swap;\n src: url(/_mintlify/assets/fonts/google/s/manrope/v23/xn7gYHE41ni1AdIRggmxSuXd.woff2) format('woff2');\n unicode-range: U+0100-02BA, U+02BD-02C5, U+02C7-02CC, U+02CE-02D7, U+02DD-02FF, U+0304, U+0308, U+0329, U+1D00-1DBF, U+1E00-1E9F, U+1EF2-1EFF, U+2020, U+20A0-20AB, U+20AD-20C4, U+2113, U+2C60-2C7F, U+A720-A7FF;\n}\n/* latin */\n@font-face {\n font-family: 'Manrope';\n font-style: normal;\n font-weight: 500;\n font-display: swap;\n src: url(/_mintlify/assets/fonts/google/s/manrope/v23/xn7gYHE41ni1AdIRggexSg.woff2) format('woff2');\n unicode-range: U+0000-00FF, U+0131, U+0152-0153, U+02BB-02BC, U+02C6, U+02DA, U+02DC, U+0304, U+0308, U+0329, U+2000-206F, U+20AC, U+2122, U+2191, U+2193, U+2212, U+2215, U+FEFF, U+FFFD;\n}\n/* cyrillic-ext */\n@font-face {\n font-family: 'Manrope';\n font-style: normal;\n font-weight: 600;\n font-display: swap;\n src: url(/_mintlify/assets/fonts/google/s/manrope/v23/xn7gYHE41ni1AdIRggqxSuXd.woff2) format('woff2');\n unicode-range: U+0460-052F, U+1C80-1C8A, U+20B4, U+2DE0-2DFF, U+A640-A69F, U+FE2E-FE2F;\n}\n/* cyrillic */\n@font-face {\n font-family: 'Manrope';\n font-style: normal;\n font-weight: 600;\n font-display: swap;\n src: url(/_mintlify/assets/fonts/google/s/manrope/v23/xn7gYHE41ni1AdIRggOxSuXd.woff2) format('woff2');\n unicode-range: U+0301, U+0400-045F, U+0490-0491, U+04B0-04B1, U+2116;\n}\n/* greek */\n@font-face {\n font-family: 'Manrope';\n font-style: normal;\n font-weight: 600;\n font-display: swap;\n src: url(/_mintlify/assets/fonts/google/s/manrope/v23/xn7gYHE41ni1AdIRggSxSuXd.woff2) format('woff2');\n unicode-range: U+0370-0377, U+037A-037F, U+0384-038A, U+038C, U+038E-03A1, U+03A3-03FF;\n}\n/* vietnamese */\n@font-face {\n font-family: 'Manrope';\n font-style: normal;\n font-weight: 600;\n font-display: swap;\n src: url(/_mintlify/assets/fonts/google/s/manrope/v23/xn7gYHE41ni1AdIRggixSuXd.woff2) format('woff2');\n unicode-range: U+0102-0103, U+0110-0111, U+0128-0129, U+0168-0169, U+01A0-01A1, U+01AF-01B0, U+0300-0301, U+0303-0304, U+0308-0309, U+0323, U+0329, U+1EA0-1EF9, U+20AB;\n}\n/* latin-ext */\n@font-face {\n font-family: 'Manrope';\n font-style: normal;\n font-weight: 600;\n font-display: swap;\n src: url(/_mintlify/assets/fonts/google/s/manrope/v23/xn7gYHE41ni1AdIRggmxSuXd.woff2) format('woff2');\n unicode-range: U+0100-02BA, U+02BD-02C5, U+02C7-02CC, U+02CE-02D7, U+02DD-02FF, U+0304, U+0308, U+0329, U+1D00-1DBF, U+1E00-1E9F, U+1EF2-1EFF, U+2020, U+20A0-20AB, U+20AD-20C4, U+2113, U+2C60-2C7F, U+A720-A7FF;\n}\n/* latin */\n@font-face {\n font-family: 'Manrope';\n font-style: normal;\n font-weight: 600;\n font-display: swap;\n src: url(/_mintlify/assets/fonts/google/s/manrope/v23/xn7gYHE41ni1AdIRggexSg.woff2) format('woff2');\n unicode-range: U+0000-00FF, U+0131, U+0152-0153, U+02BB-02BC, U+02C6, U+02DA, U+02DC, U+0304, U+0308, U+0329, U+2000-206F, U+20AC, U+2122, U+2191, U+2193, U+2212, U+2215, U+FEFF, U+FFFD;\n}\n/* cyrillic-ext */\n@font-face {\n font-family: 'Manrope';\n font-style: normal;\n font-weight: 700;\n font-display: swap;\n src: url(/_mintlify/assets/fonts/google/s/manrope/v23/xn7gYHE41ni1AdIRggqxSuXd.woff2) format('woff2');\n unicode-range: U+0460-052F, U+1C80-1C8A, U+20B4, U+2DE0-2DFF, U+A640-A69F, U+FE2E-FE2F;\n}\n/* cyrillic */\n@font-face {\n font-family: 'Manrope';\n font-style: normal;\n font-weight: 700;\n font-display: swap;\n src: url(/_mintlify/assets/fonts/google/s/manrope/v23/xn7gYHE41ni1AdIRggOxSuXd.woff2) format('woff2');\n unicode-range: U+0301, U+0400-045F, U+0490-0491, U+04B0-04B1, U+2116;\n}\n/* greek */\n@font-face {\n font-family: 'Manrope';\n font-style: normal;\n font-weight: 700;\n font-display: swap;\n src: url(/_mintlify/assets/fonts/google/s/manrope/v23/xn7gYHE41ni1AdIRggSxSuXd.woff2) format('woff2');\n unicode-range: U+0370-0377, U+037A-037F, U+0384-038A, U+038C, U+038E-03A1, U+03A3-03FF;\n}\n/* vietnamese */\n@font-face {\n font-family: 'Manrope';\n font-style: normal;\n font-weight: 700;\n font-display: swap;\n src: url(/_mintlify/assets/fonts/google/s/manrope/v23/xn7gYHE41ni1AdIRggixSuXd.woff2) format('woff2');\n unicode-range: U+0102-0103, U+0110-0111, U+0128-0129, U+0168-0169, U+01A0-01A1, U+01AF-01B0, U+0300-0301, U+0303-0304, U+0308-0309, U+0323, U+0329, U+1EA0-1EF9, U+20AB;\n}\n/* latin-ext */\n@font-face {\n font-family: 'Manrope';\n font-style: normal;\n font-weight: 700;\n font-display: swap;\n src: url(/_mintlify/assets/fonts/google/s/manrope/v23/xn7gYHE41ni1AdIRggmxSuXd.woff2) format('woff2');\n unicode-range: U+0100-02BA, U+02BD-02C5, U+02C7-02CC, U+02CE-02D7, U+02DD-02FF, U+0304, U+0308, U+0329, U+1D00-1DBF, U+1E00-1E9F, U+1EF2-1EFF, U+2020, U+20A0-20AB, U+20AD-20C4, U+2113, U+2C60-2C7F, U+A720-A7FF;\n}\n/* latin */\n@font-face {\n font-family: 'Manrope';\n font-style: normal;\n font-weight: 700;\n font-display: swap;\n src: url(/_mintlify/assets/fonts/google/s/manrope/v23/xn7gYHE41ni1AdIRggexSg.woff2) format('woff2');\n unicode-range: U+0000-00FF, U+0131, U+0152-0153, U+02BB-02BC, U+02C6, U+02DA, U+02DC, U+0304, U+0308, U+0329, U+2000-206F, U+20AC, U+2122, U+2191, U+2193, U+2212, U+2215, U+FEFF, U+FFFD;\n}\n/* latin-ext */\n@font-face {\n font-family: 'Sometype Mono';\n font-style: italic;\n font-weight: 400 700;\n font-display: swap;\n src: url(/_mintlify/assets/fonts/google/s/sometypemono/v4/70lLu745KGk_R3uxyq0WrROhKpGRTW9Gzm8.woff2) format('woff2');\n unicode-range: U+0100-02BA, U+02BD-02C5, U+02C7-02CC, U+02CE-02D7, U+02DD-02FF, U+0304, U+0308, U+0329, U+1D00-1DBF, U+1E00-1E9F, U+1EF2-1EFF, U+2020, U+20A0-20AB, U+20AD-20C4, U+2113, U+2C60-2C7F, U+A720-A7FF;\n}\n/* latin */\n@font-face {\n font-family: 'Sometype Mono';\n font-style: italic;\n font-weight: 400 700;\n font-display: swap;\n src: url(/_mintlify/assets/fonts/google/s/sometypemono/v4/70lLu745KGk_R3uxyq0WrROhKpGRQ29G.woff2) format('woff2');\n unicode-range: U+0000-00FF, U+0131, U+0152-0153, U+02BB-02BC, U+02C6, U+02DA, U+02DC, U+0304, U+0308, U+0329, U+2000-206F, U+20AC, U+2122, U+2191, U+2193, U+2212, U+2215, U+FEFF, U+FFFD;\n}\n/* latin-ext */\n@font-face {\n font-family: 'Sometype Mono';\n font-style: normal;\n font-weight: 400 700;\n font-display: swap;\n src: url(/_mintlify/assets/fonts/google/s/sometypemono/v4/70lVu745KGk_R3uxyq0WrROhKpqhQXdC.woff2) format('woff2');\n unicode-range: U+0100-02BA, U+02BD-02C5, U+02C7-02CC, U+02CE-02D7, U+02DD-02FF, U+0304, U+0308, U+0329, U+1D00-1DBF, U+1E00-1E9F, U+1EF2-1EFF, U+2020, U+20A0-20AB, U+20AD-20C4, U+2113, U+2C60-2C7F, U+A720-A7FF;\n}\n/* latin */\n@font-face {\n font-family: 'Sometype Mono';\n font-style: normal;\n font-weight: 400 700;\n font-display: swap;\n src: url(/_mintlify/assets/fonts/google/s/sometypemono/v4/70lVu745KGk_R3uxyq0WrROhKpShQQ.woff2) format('woff2');\n unicode-range: U+0000-00FF, U+0131, U+0152-0153, U+02BB-02BC, U+02C6, U+02DA, U+02DC, U+0304, U+0308, U+0329, U+2000-206F, U+20AC, U+2122, U+2191, U+2193, U+2212, U+2215, U+FEFF, U+FFFD;\n}\n#navbar .h-6{height:4.5rem;padding-top:3px;padding-bottom:3px}:root{--cerebras-brand-50: #F1592A;--cerebras-brand-55: #C0441F;--cerebras-interactive-10: #F2F9FB;--cerebras-interactive-50: #F1592A;--cerebras-interactive-55: #C0441F;--cerebras-neutral-90: #1B1A1E;--cerebras-neutral-95: #161519;--cerebras-neutral-15: #EAEAEB;--cerebras-neutral-60: #424049;--cerebras-shadow-soft: 0px 1px 4px 0px #0000000F;--cerebras-shadow-soft-md: 0px 2px 6px 1px #0000000F}html,body{-webkit-font-smoothing:antialiased;-moz-osx-font-smoothing:grayscale;text-rendering:geometricPrecision;font-family:'Manrope', ui-sans-serif, system-ui, -apple-system,\n BlinkMacSystemFont, 'Segoe UI', Roboto, 'Helvetica Neue', Arial,\n 'Apple Color Emoji', 'Segoe UI Emoji', 'Segoe UI Symbol', 'Noto Color Emoji'}link{color:inherit;text-decoration-color:var(--cerebras-brand-55)}link:hover{color:var(--cerebras-brand-50);text-decoration-color:var(--cerebras-brand-50)}#content-area :is(p,li,td,th,blockquote) a:focus-visible{outline:2px dashed var(--cerebras-brand-55);outline-offset:2px}#sidebar-title{text-transform:uppercase!important;font-family:\"Sometype Mono\"}#content-area{max-width:80ch;margin:0 auto;font-size:0.95rem;line-height:1.55}#content-area table{width:100%;color:var(--cerebras-neutral-95);background:#FFFFFF;border:1px solid var(--cerebras-neutral-15);border-radius:0.75rem;box-shadow:var(--cerebras-shadow-soft);overflow:hidden}#content-area thead tr{border-bottom:2px solid var(--cerebras-neutral-15)}#content-area tbody tr{border-bottom:1px solid var(--cerebras-neutral-15)}#content-area tbody tr:last-child{border-bottom:0}#content-area :is(th,td){padding:0.5rem;vertical-align:top}html[style*=\"color-scheme: dark\"] #content-area table,html[style*=\"color-scheme:dark\"] #content-area table{color:var(--cerebras-neutral-15);background:var(--cerebras-neutral-95);border-color:var(--cerebras-neutral-60)}html[style*=\"color-scheme: dark\"] #content-area thead tr,html[style*=\"color-scheme:dark\"] #content-area thead tr{border-bottom-color:var(--cerebras-neutral-60)}html[style*=\"color-scheme: dark\"] #content-area tbody tr,html[style*=\"color-scheme:dark\"] #content-area tbody tr{border-bottom-color:var(--cerebras-neutral-60)}#content-area td:first-child{font-weight:600}#content-area :is(h1,h2,h3){letter-spacing:-0.02em}#content-area :is(h1,h2){font-weight:600}#content-area :is(h3,h4){font-weight:600}#table-of-contents-content{text-transform:uppercase!important;font-family:\"Sometype Mono\"}.bg-\\[\\#F4F4F5\\]{--tw-bg-opacity: 1;background-color:#EBEBEB}"}]}
{"page":{"pagePath":"rel-2.6.0/concepts/cerebras-wafer-scale-cluster","slug":"/rel-2.6.0/concepts/cerebras-wafer-scale-cluster","pageMetadata":{"title":"Cerebras Wafer Scale Cluster","description":null,"href":"/rel-2.6.0/concepts/cerebras-wafer-scale-cluster"},"pageType":"default","mdx":{"mdxSource":{"compiledSource":"\"use strict\";const{Fragment:_Fragment,jsx:_jsx,jsxs:_jsxs}=arguments[0];const{useMDXComponents:_provideComponents}=arguments[0];function _createMdxContent(props){const _components={a:`a`,p:`p`,..._provideComponents(),...props.components},{Frame,Heading,OptimizedImage}=_components;if(!Frame)_missingMdxReference(`Frame`,true);if(!Heading)_missingMdxReference(`Heading`,true);if(!OptimizedImage)_missingMdxReference(`OptimizedImage`,true);return _jsxs(_Fragment,{children:[_jsxs(_components.p,{children:[`The Cerebras `,_jsx(_components.a,{href:`https://www.cerebras.net/product-cluster/`,children:`Wafer-Scale Cluster`}),` is meticulously engineered to enable the training of neural networks with remarkably efficient linear scaling across millions of cores, all without the complexities of traditional distributed computing.`]}),`\n`,_jsx(Frame,{caption:`Fig. 1: Topology of Cerebras Wafer-Scale Cluster`,children:_jsx(OptimizedImage,{src:`https://mintcdn.com/cerebras-training/NBvBobn0ysIp4rEe/rel-2.6.0/images/concepts/topology-of-weight-streaming-on-wsc.png?fit=max\u0026auto=format\u0026n=NBvBobn0ysIp4rEe\u0026q=85\u0026s=88b497e3b5ecc11d1d03cf770eeb5e34`,\"data-og-width\":`970`,width:`970`,\"data-og-height\":`418`,height:`418`,\"data-path\":`rel-2.6.0/images/concepts/topology-of-weight-streaming-on-wsc.png`,\"data-optimize\":`true`,\"data-opv\":`3`,srcset:`https://mintcdn.com/cerebras-training/NBvBobn0ysIp4rEe/rel-2.6.0/images/concepts/topology-of-weight-streaming-on-wsc.png?w=280\u0026fit=max\u0026auto=format\u0026n=NBvBobn0ysIp4rEe\u0026q=85\u0026s=99c2f01e26501fe614b592050ef54a2c 280w, https://mintcdn.com/cerebras-training/NBvBobn0ysIp4rEe/rel-2.6.0/images/concepts/topology-of-weight-streaming-on-wsc.png?w=560\u0026fit=max\u0026auto=format\u0026n=NBvBobn0ysIp4rEe\u0026q=85\u0026s=01322613b7718d4c5189f2b11aaf8ed6 560w, https://mintcdn.com/cerebras-training/NBvBobn0ysIp4rEe/rel-2.6.0/images/concepts/topology-of-weight-streaming-on-wsc.png?w=840\u0026fit=max\u0026auto=format\u0026n=NBvBobn0ysIp4rEe\u0026q=85\u0026s=f7bd26f56a265c619398cb0027683f69 840w, https://mintcdn.com/cerebras-training/NBvBobn0ysIp4rEe/rel-2.6.0/images/concepts/topology-of-weight-streaming-on-wsc.png?w=1100\u0026fit=max\u0026auto=format\u0026n=NBvBobn0ysIp4rEe\u0026q=85\u0026s=8f3818c8c49e86d0734daeea75c3292e 1100w, https://mintcdn.com/cerebras-training/NBvBobn0ysIp4rEe/rel-2.6.0/images/concepts/topology-of-weight-streaming-on-wsc.png?w=1650\u0026fit=max\u0026auto=format\u0026n=NBvBobn0ysIp4rEe\u0026q=85\u0026s=8ab84a2709776cda714d3994fc3f951f 1650w, https://mintcdn.com/cerebras-training/NBvBobn0ysIp4rEe/rel-2.6.0/images/concepts/topology-of-weight-streaming-on-wsc.png?w=2500\u0026fit=max\u0026auto=format\u0026n=NBvBobn0ysIp4rEe\u0026q=85\u0026s=ff354510bc1943bb9e77a4f0f292ec83 2500w`})}),`\n`,_jsx(Heading,{level:`2`,id:`key-components`,children:`Key Components`}),`\n`,_jsx(Heading,{level:`3`,id:`cs-3-systems`,children:`CS-3 Systems`}),`\n`,_jsxs(_components.p,{children:[`The `,_jsx(_components.a,{href:`https://www.cerebras.net/product-system/`,children:`CS-3 systems`}),` represents the latest advancement in our lineup, incorporating the innovative WSE-3 (Wafer-Scale Engine 3) to deliver cutting-edge performance. Building on the user-friendly experience of its predecessor, the CS-2, the CS-3 significantly enhances computational efficiency and performance.`]}),`\n`,_jsx(_components.p,{children:`This next-gen system introduces a novel instruction set architecture and boosts the SIMD (Single Instruction, Multiple Data) width, providing more parallel processing power. Additionally, it expands the core count by 20% and enhances the ingress/egress bandwidth by 30%, enabling faster data processing and transfer rates.`}),`\n`,_jsxs(_components.p,{children:[`Compatibility and performance testing are a cornerstone of the CS-3’s development. All models available in the Model Zoo have been verified for functionality on the CS-3 platform. Furthermore, a selection of key models, including GPT-2, GPT-3, LLaMA, Falcon, BTLM, GPT-J, GPT-NeoX, T5, BERT, and PaLI have undergone rigorous testing to ensure both convergence and performance are optimized for the CS-3, guaranteeing a seamless and efficient user experience. You can find more detailed information in the `,_jsx(_components.a,{href:`https://8968533.fs1.hubspotusercontent-na1.net/hubfs/8968533/Datasheets/WSE-3%20Datasheet.pdf`,children:`WSE-3 datasheet`}),` and explore `,_jsx(_components.a,{href:`https://www.cerebras.net/product-system/`,children:`CS-3`}),`.`]}),`\n`,_jsx(Heading,{level:`3`,id:`memoryx-technology`,children:`MemoryX Technology`}),`\n`,_jsxs(_components.p,{children:[_jsx(_components.a,{href:`https://www.cerebras.net/blog/announcing-the-cerebras-architecture-for-extreme-scale-ai/`,children:`MemoryX technology`}),` serves as the storage and intelligent streaming solution for a model’s weights, ensuring efficient and timely access for the CS-X systems.`]}),`\n`,_jsx(Heading,{level:`3`,id:`swarmx-technology`,children:`SwarmX Technology`}),`\n`,_jsxs(_components.p,{children:[_jsx(_components.a,{href:`https://www.cerebras.net/blog/announcing-the-cerebras-architecture-for-extreme-scale-ai/`,children:`SwarmX technology`}),` plays a pivotal role in integrating multiple CS-X nodes into a unified Cerebras cluster. These nodes collaborate seamlessly in training a single model. SwarmX handles the broadcast of weights from MemoryX to the entire cluster and effectively reduces (sums) gradients in the opposite direction, contributing to efficient training processes.`]}),`\n`,_jsx(Heading,{level:`3`,id:`input-preprocessing-servers`,children:`Input Preprocessing Servers`}),`\n`,_jsx(_components.p,{children:`Input Preprocessing servers handle the critical task of preprocessing training data, ensuring that it is appropriately prepared before being dispatched to the CS-X systems. This preprocessing step is vital for training, inference, and evaluation.`}),`\n`,_jsx(Heading,{level:`3`,id:`management-servers`,children:`Management Servers`}),`\n`,_jsx(_components.p,{children:`Management servers are responsible for orchestrating and scheduling the cluster’s resources, ensuring efficient utilization and coordination among all components in the Cerebras cluster. They play a key role in optimizing the cluster’s performance and resource allocation.`}),`\n`,_jsx(Heading,{level:`2`,id:`working-with-the-cerebras-cluster`,children:`Working with the Cerebras Cluster`}),`\n`,_jsx(_components.p,{children:`After developing your code, you can initiate the process of submitting it for both training and evaluation from a user node. It’s important to note that the user node operates independently from the cluster and connects to the Cerebras cluster through the management server, as illustrated in Fig. 1 above. The management server is responsible for handling all resource scheduling. Therefore, your main task is to specify the number of CS-X systems you wish to allocate for either training or evaluation. This enables you to efficiently utilize the Cerebras cluster’s resources without directly managing the intricacies of cluster allocation.`}),`\n`,_jsx(_components.p,{children:`For documentation related to the installation and administration of the Cerebras Wafer-Scale cluster, visit Cerebras deployment documentation.`})]})}function MDXContent(props={}){const{wrapper:MDXLayout}={..._provideComponents(),...props.components};return MDXLayout?_jsx(MDXLayout,{...props,children:_jsx(_createMdxContent,{...props})}):_createMdxContent(props)}return{default:MDXContent};function _missingMdxReference(id,component){throw new Error(`Expected `+(component?`component`:`object`)+\" `\"+id+\"` to be defined: you likely forgot to import, pass, or provide it.\")}","frontmatter":{},"scope":{"config":{},"pageMetadata":{"title":"Cerebras Wafer Scale Cluster","description":null,"href":"/rel-2.6.0/concepts/cerebras-wafer-scale-cluster"}}},"mdxSourceWithNoJs":null},"mdxExtracts":{"tableOfContents":[{"title":"Key Components","slug":"key-components","depth":2,"children":[{"title":"CS-3 Systems","slug":"cs-3-systems","depth":3,"children":[]},{"title":"MemoryX Technology","slug":"memoryx-technology","depth":3,"children":[]},{"title":"SwarmX Technology","slug":"swarmx-technology","depth":3,"children":[]},{"title":"Input Preprocessing Servers","slug":"input-preprocessing-servers","depth":3,"children":[]},{"title":"Management Servers","slug":"management-servers","depth":3,"children":[]}]},{"title":"Working with the Cerebras Cluster","slug":"working-with-the-cerebras-cluster","depth":2,"children":[]}],"codeExamples":{}},"jsLinks":[],"subdomain":"cerebras-training","siteName":"Cerebras Training","title":"Cerebras Wafer Scale Cluster - Cerebras Training","navMaps":{"firstHrefInVersion":{"rel-2.5.0":{"href":"/rel-2.5.0/concepts/cerebras-wafer-scale-cluster","matchLevel":1},"rel-2.10.0-beta":{"href":"/rel-2.10.0/concepts/cerebras-wafer-scale-cluster","matchLevel":1},"rel-2.9.0-beta":{"href":"/rel-2.9.0/concepts/cerebras-wafer-scale-cluster","matchLevel":1},"rel-2.8.0-beta":{"href":"/rel-2.8.0/concepts/cerebras-wafer-scale-cluster","matchLevel":1},"rel-2.7.0-beta":{"href":"/rel-2.7.0/concepts/cerebras-wafer-scale-cluster","matchLevel":1},"rel-2.6.0-beta":{"href":"/rel-2.6.0/concepts/cerebras-wafer-scale-cluster","matchLevel":0},"rel-2.4.0":{"href":"/rel-2.4.0/concepts/cerebras-wafer-scale-cluster","matchLevel":1}},"firstHrefInLanguage":{}},"lastModified":"2026-06-15T17:54:53.994Z","originalFileLocation":"rel-2.6.0/concepts/cerebras-wafer-scale-cluster.mdx"},"headHtml":"\u003cscript data-mint-head>(function(){var d=document.documentElement,p=null;if(!p){try{p=localStorage.getItem(\"isDarkMode\")}catch(e){}if(p==='true')p='dark';else if(p==='false')p='light';if(p!=='light'\u0026\u0026p!=='dark'\u0026\u0026p!=='system')p=\"system\"}var r=p==='system'?(matchMedia(\"(prefers-color-scheme: dark)\").matches?'dark':'light'):p;d.classList.remove(r==='dark'?'light':'dark');d.classList.add(r);d.setAttribute(\"data-theme-preference\",p)})();\u003c/script>\u003cmeta data-mint-head name=\"application-name\" content=\"Cerebras Training\">\u003cmeta data-mint-head name=\"generator\" content=\"Mintlify\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"192x192\" href=\"/mintlify-assets/_mintlify/favicons/cerebras-training/kLGG7BxUKv2Tig4k/_generated/favicon/android-chrome-192x192.png\" media=\"(prefers-color-scheme: light)\">\u003clink data-mint-head rel=\"apple-touch-icon\" type=\"image/png\" sizes=\"180x180\" href=\"/mintlify-assets/_mintlify/favicons/cerebras-training/kLGG7BxUKv2Tig4k/_generated/favicon/apple-touch-icon.png\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"16x16\" href=\"/mintlify-assets/_mintlify/favicons/cerebras-training/kLGG7BxUKv2Tig4k/_generated/favicon/favicon-16x16.png\" media=\"(prefers-color-scheme: light)\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"32x32\" href=\"/mintlify-assets/_mintlify/favicons/cerebras-training/kLGG7BxUKv2Tig4k/_generated/favicon/favicon-32x32.png\" media=\"(prefers-color-scheme: light)\">\u003clink data-mint-head rel=\"shortcut icon\" type=\"image/x-icon\" href=\"/mintlify-assets/_mintlify/favicons/cerebras-training/kLGG7BxUKv2Tig4k/_generated/favicon/favicon.ico\" media=\"(prefers-color-scheme: light)\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"192x192\" href=\"/mintlify-assets/_mintlify/favicons/cerebras-training/kLGG7BxUKv2Tig4k/_generated/favicon-dark/android-chrome-192x192.png\" media=\"(prefers-color-scheme: dark)\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"16x16\" href=\"/mintlify-assets/_mintlify/favicons/cerebras-training/kLGG7BxUKv2Tig4k/_generated/favicon-dark/favicon-16x16.png\" media=\"(prefers-color-scheme: dark)\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"32x32\" href=\"/mintlify-assets/_mintlify/favicons/cerebras-training/kLGG7BxUKv2Tig4k/_generated/favicon-dark/favicon-32x32.png\" media=\"(prefers-color-scheme: dark)\">\u003clink data-mint-head rel=\"shortcut icon\" type=\"image/x-icon\" href=\"/mintlify-assets/_mintlify/favicons/cerebras-training/kLGG7BxUKv2Tig4k/_generated/favicon-dark/favicon.ico\" media=\"(prefers-color-scheme: dark)\">\u003cmeta data-mint-head property=\"og:title\" content=\"Cerebras Wafer Scale Cluster - Cerebras Training\">\u003cmeta data-mint-head property=\"og:site_name\" content=\"Cerebras Training\">\u003cmeta data-mint-head property=\"og:url\" content=\"https://training-docs.cerebras.ai/rel-2.6.0/concepts/cerebras-wafer-scale-cluster\">\u003cmeta data-mint-head property=\"og:type\" content=\"website\">\u003cmeta data-mint-head property=\"og:image\" content=\"https://cerebras-training.mintlify.site/_mintlify/api/og/v2?division=Explore+%26+Learn\u0026amp;title=Cerebras+Wafer+Scale+Cluster\u0026amp;theme=dc7fc330e54df6659e7a2b3e\u0026amp;signature=b4f3acecef5d3f8e86b5a7cf63007ad213c7110d33b19ab15b62c227b068f1bd\">\u003cmeta data-mint-head property=\"og:image:width\" content=\"1200\">\u003cmeta data-mint-head property=\"og:image:height\" content=\"630\">\u003cmeta data-mint-head name=\"twitter:card\" content=\"summary_large_image\">\u003cmeta data-mint-head name=\"twitter:title\" content=\"Cerebras Wafer Scale Cluster - Cerebras Training\">\u003cmeta data-mint-head name=\"twitter:image\" content=\"https://cerebras-training.mintlify.site/_mintlify/api/og/v2?division=Explore+%26+Learn\u0026amp;title=Cerebras+Wafer+Scale+Cluster\u0026amp;theme=dc7fc330e54df6659e7a2b3e\u0026amp;signature=b4f3acecef5d3f8e86b5a7cf63007ad213c7110d33b19ab15b62c227b068f1bd\">\u003cmeta data-mint-head name=\"twitter:image:width\" content=\"1200\">\u003cmeta data-mint-head name=\"twitter:image:height\" content=\"630\">\u003cmeta data-mint-head name=\"msapplication-config\" content=\"/mintlify-assets/_mintlify/favicons/cerebras-training/kLGG7BxUKv2Tig4k/_generated/favicon/browserconfig.xml\">\u003cmeta data-mint-head name=\"apple-mobile-web-app-title\" content=\"Cerebras Training\">\u003cmeta data-mint-head name=\"msapplication-TileColor\" content=\"#F1592A\">\u003clink data-mint-head rel=\"canonical\" href=\"https://training-docs.cerebras.ai/rel-2.6.0/concepts/cerebras-wafer-scale-cluster\">\u003clink data-mint-head rel=\"alternate\" type=\"application/xml\" href=\"/sitemap.xml\">\u003clink data-mint-head rel=\"alternate\" type=\"text/markdown\" href=\"/rel-2.6.0/concepts/cerebras-wafer-scale-cluster.md\">\u003cscript data-mint-head type=\"application/ld+json\">{\"@context\":\"https://schema.org\",\"@graph\":[{\"@type\":\"Organization\",\"@id\":\"https://training-docs.cerebras.ai/#organization\",\"name\":\"Cerebras Training\",\"url\":\"https://training-docs.cerebras.ai\",\"logo\":{\"@type\":\"ImageObject\",\"url\":\"https://mintcdn.com/cerebras-training/hh6MqoQlI8YKsSkL/logo/2.svg?fit=max\u0026auto=format\u0026n=hh6MqoQlI8YKsSkL\u0026q=85\u0026s=9430343b4d9c7548560af71291450f1f\"}},{\"@type\":\"WebSite\",\"@id\":\"https://training-docs.cerebras.ai#website\",\"name\":\"Cerebras Training\",\"url\":\"https://training-docs.cerebras.ai\",\"publisher\":{\"@id\":\"https://training-docs.cerebras.ai/#organization\"}},{\"@type\":\"WebPage\",\"@id\":\"https://training-docs.cerebras.ai/rel-2.6.0/concepts/cerebras-wafer-scale-cluster#webpage\",\"url\":\"https://training-docs.cerebras.ai/rel-2.6.0/concepts/cerebras-wafer-scale-cluster\",\"name\":\"Cerebras Wafer Scale Cluster\",\"dateModified\":\"2026-06-15T17:54:53.994Z\",\"isPartOf\":{\"@id\":\"https://training-docs.cerebras.ai#website\"},\"breadcrumb\":{\"@id\":\"https://training-docs.cerebras.ai/rel-2.6.0/concepts/cerebras-wafer-scale-cluster#breadcrumb\"}},{\"@type\":\"BreadcrumbList\",\"@id\":\"https://training-docs.cerebras.ai/rel-2.6.0/concepts/cerebras-wafer-scale-cluster#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Cerebras Training\",\"item\":\"https://training-docs.cerebras.ai\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Cerebras Wafer Scale Cluster\",\"item\":\"https://training-docs.cerebras.ai/rel-2.6.0/concepts/cerebras-wafer-scale-cluster\"}]},{\"@type\":[\"Article\",\"TechArticle\"],\"@id\":\"https://training-docs.cerebras.ai/rel-2.6.0/concepts/cerebras-wafer-scale-cluster#article\",\"headline\":\"Cerebras Wafer Scale Cluster\",\"name\":\"Cerebras Wafer Scale Cluster\",\"url\":\"https://training-docs.cerebras.ai/rel-2.6.0/concepts/cerebras-wafer-scale-cluster\",\"mainEntityOfPage\":{\"@id\":\"https://training-docs.cerebras.ai/rel-2.6.0/concepts/cerebras-wafer-scale-cluster#webpage\"},\"dateModified\":\"2026-06-15T17:54:53.994Z\",\"publisher\":{\"@id\":\"https://training-docs.cerebras.ai/#organization\"},\"isPartOf\":{\"@id\":\"https://training-docs.cerebras.ai#website\"}}]}\u003c/script>","agentPointerHref":"/rel-2.6.0/concepts/cerebras-wafer-scale-cluster.md","siteName":"Cerebras Training","nav":{"global":{"anchors":[{"anchor":"Community","icon":"discord","href":"https://discord.com/invite/ZqvYS2e2rY"},{"anchor":"API Reference","icon":"code","href":"https://training-api.cerebras.ai"}]},"versions":[{"version":"rel-2.5.0","pages":[{"title":"rel-2.5.0","href":"/rel-2.5.0/concepts/cerebras-wafer-scale-cluster"}]},{"version":"rel-2.10.0-beta","pages":[{"title":"rel-2.10.0-beta","href":"/rel-2.10.0/concepts/cerebras-wafer-scale-cluster"}]},{"version":"rel-2.9.0-beta","pages":[{"title":"rel-2.9.0-beta","href":"/rel-2.9.0/concepts/cerebras-wafer-scale-cluster"}]},{"version":"rel-2.8.0-beta","pages":[{"title":"rel-2.8.0-beta","href":"/rel-2.8.0/concepts/cerebras-wafer-scale-cluster"}]},{"version":"rel-2.7.0-beta","pages":[{"title":"rel-2.7.0-beta","href":"/rel-2.7.0/concepts/cerebras-wafer-scale-cluster"}]},{"version":"rel-2.6.0-beta","tabs":[{"tab":"Guides","pages":[{"title":"Guides","href":"/rel-2.6.0/getting-started/overview"}]},{"tab":"Explore \u0026 Learn","groups":[{"group":"Explore \u0026 Learn","pages":[{"title":"Cerebras Wafer Scale Cluster","href":"/rel-2.6.0/concepts/cerebras-wafer-scale-cluster"},{"title":"Weight Streaming Execution","description":null,"href":"/rel-2.6.0/concepts/weight-streaming-execution"},{"group":"Writing a Custom Training Loop","pages":[{"title":"Writing a Custom Training Loop","href":"/rel-2.6.0/cs-torch/writing-a-custom-training-loop"},{"title":"Step Closures","href":"/rel-2.6.0/cs-torch/writing-a-custom-training-loop/step-closures"},{"title":"Saving Loading Checkpoint","href":"/rel-2.6.0/cs-torch/writing-a-custom-training-loop/saving-loading-checkpoints"},{"title":"Efficient Weight Initialization ","href":"/rel-2.6.0/cs-torch/writing-a-custom-training-loop/efficient-weight-initialization"},{"title":"Writing Custom Optimizers ","href":"/rel-2.6.0/cs-torch/writing-a-custom-training-loop/writing-custom-optimizers"},{"title":"Learning Rate Scheduling","href":"/rel-2.6.0/cs-torch/writing-a-custom-training-loop/learning-rate-scheduling"},{"title":"Gradient Scaling","href":"/rel-2.6.0/cs-torch/writing-a-custom-training-loop/gradient-scaling"},{"title":"Sparsifying Models ","href":"/rel-2.6.0/cs-torch/writing-a-custom-training-loop/sparsifying-models"},{"title":"Restartable Dataloaders ","href":"/rel-2.6.0/cs-torch/writing-a-custom-training-loop/restartable-dataloaders"},{"title":"Profiling the Executor ","href":"/rel-2.6.0/cs-torch/writing-a-custom-training-loop/profiling-the-executor"},{"title":"Evaluation Metrics ","href":"/rel-2.6.0/cs-torch/writing-a-custom-training-loop/evaluation-metrics"},{"title":"Per Layer Precision Optimization Level ","href":"/rel-2.6.0/cs-torch/writing-a-custom-training-loop/per-layer-precision-optimization-level"},{"title":"Limitations of Pytorch on Cerebras","href":"/rel-2.6.0/cs-torch/writing-a-custom-training-loop/limitations-of-pytorch-on-cerebras"}]},{"group":"Tutorials","pages":[{"title":"Extend Context Length Using Position Interpolation","href":"/rel-2.6.0/model-zoo/tutorials/extend-context-length-using-position-interpolation"},{"title":"Train an LLM with a Large or Small Context Window","href":"/rel-2.6.0/model-zoo/tutorials/train-an-llm-with-a-large-or-small-context-window"},{"title":"Instruction Fine-Tuning for LLMs","href":"/rel-2.6.0/model-zoo/tutorials/instruction-fine-tuning-for-llms"},{"title":"Train a Model with Weight Sparsity","href":"/rel-2.6.0/model-zoo/tutorials/train-a-model-with-weight-sparsity"},{"group":"Microbatching","pages":[{"title":"Automatic Microbatching","href":"/rel-2.6.0/model-zoo/tutorials/optimize-performance-with-automatic-microbatching"},{"title":"Beginner: Automatic Microbatching","sidebarTitle":"Beginner Guide","href":"/rel-2.6.0/model-zoo/tutorials/microbatching-beginner"},{"title":"Advanced: Understanding and Manually Controlling MBS","sidebarTitle":"Advanced Guide","href":"/rel-2.6.0/model-zoo/tutorials/microbatching-advanced"}]},{"group":"Train LLMs with Maximal Update Parameterization","pages":[{"title":"Train LLMs with μP","href":"/rel-2.6.0/model-zoo/tutorials/training-an-llm-using-maximal-update-parameterization/overview"},{"title":"Configure μP for GPT Style Models","href":"/rel-2.6.0/model-zoo/tutorials/training-an-llm-using-maximal-update-parameterization/configure-mp-for-gpt-style-models"},{"title":"Configure μP for Bert Pretrain Beta","href":"/rel-2.6.0/model-zoo/tutorials/training-an-llm-using-maximal-update-parameterization/configure-mp-for-bert-pretrain--beta"},{"title":"Configure μP for T5 Beta","href":"/rel-2.6.0/model-zoo/tutorials/training-an-llm-using-maximal-update-parameterization/configure-mp-for-t5--beta"}]},{"title":"Train LLMs with μP: Legacy Parameters","href":"/rel-2.6.0/model-zoo/tutorials/train-an-llm-using-maximal-update-parameterization-with-legacy-params"},{"title":"Dynamic Loss Scaling","href":"/rel-2.6.0/model-zoo/tutorials/dynamic-loss-scaling"},{"title":" Training With Number Of Tokens Loss Scaling ","href":"/rel-2.6.0/model-zoo/tutorials/training-with-number-of-tokens-loss-scaling"}]}]}]},{"tab":"Release Notes","pages":[{"title":"Release Notes","href":"/rel-2.6.0/release-notes/release-notes-tabs"}]}]},{"version":"rel-2.4.0","pages":[{"title":"rel-2.4.0","href":"/rel-2.4.0/concepts/cerebras-wafer-scale-cluster"}]}]},"basePath":"","title":"Cerebras Wafer Scale Cluster - Cerebras Training","deploymentId":"dpl_2pTxFtyBudBMYZiCw1MyvoA1wxpd","buildId":"dpl_2pTxFtyBudBMYZiCw1MyvoA1wxpd","dynamics":["07c022d89b","a74be98ba7"],"chunks":["Frame-B0oVBAXA.js","DynamicMarkdown-BvJIe00T.js","zoom-image-lightbox-BXthlMNf.js","lightbox-handle-r17bVUz3.js"]}