Learn how jobs are prioritized on a Cerebras cluster.
Job scheduling is determined by assigned priorities rather than submission time. Jobs are categorized into four priority buckets, ranging from P0 (highest) to P3 (lowest). Within each bucket, jobs are ranked by numerical priority values, where lower numbers indicate higher priority. Scheduling follows these rankings within designated queues.
Users can specify a job priority at submission using the --job_priority flag. For example:
Copy
Ask AI
cszoo fit params_model.yaml [additional-args] --job_priority p1
Valid inputs are p1, p2, and p3, corresponding to default values of 199, 299, and 399, respectively. If unspecified, jobs default to P2 (299). Learn more about the additional arguments for launching jobs here.
When listing jobs, you will see both priority values and their corresponding buckets. Jobs are first sorted by priority and then by age. Higher-priority jobs appear first, and within the same priority level, newer jobs are listed before older ones.
Jobs stemming from the same invocation inherit the workflow’s priority. Updating a workflow’s priority affects all associated pending and subsequent jobs, ensuring consistent priority across a workflow.
This updates the job’s priority to P2. Confirmation is displayed:
Copy
Ask AI
successfully updated wsjob-jv58f5mb95kwpe5hwuujrk job priority to p2
List jobs with their priorities:
Copy
Ask AI
$ csctl job list -a
Displays a list of jobs along with details such as type, priority, age, duration, phase, system, user, labels, and dashboard status. For example, it shows that:
Copy
Ask AI
NAME TYPE PRIORITY AGE DURATION PHASE SYSTEMS USER LABELS DASHBOARDjob-operator/wsjob-jv58f5mb95kwpe5hwuujrk execute P2 (200) 3h56m 88s SUCCEEDED kapi-1 myuser wsjob-label=test grafana not deployedjob-operator/wsjob-2ercynhxb44ajhg4qn9bem compile P1 (150) 3h58m 2m21s FAILED myuser wsjob-label=test grafana not deployed