Resources & scaling

Task sizing

Each task is dispatched to a worker sized to that task’s Nextflow resource directives. Declare them in your pipeline or a config as usual:

process STAR_ALIGN {
    cpus   12
    memory 36.GB
    disk   50.GB
}

When a task moves onto a worker, the executor resizes that worker to fit — in milliseconds, without recreating the container. A chain of stages with different sizes runs on the same warm worker, resized between each.

Your container image needs bash

Task dispatch execs into the worker’s container via bash. Minimal base images (Alpine, distroless, some biocontainers) don’t ship it by default — add it (apk add bash on Alpine) or the task fails immediately, before your process command ever runs.

Defaults for undeclared directives

If a process doesn’t declare a directive, the executor fills it in:

Directive	Default	Set via
cpus	1	Nextflow default
memory	4 GiB	ccloud.defaultMemGib
disk (scratch)	20 GiB	ccloud.defaultDiskGib

Raise defaultMemGib / defaultDiskGib in the ccloud { } block if your pipeline’s undeclared tasks need more.

The locality cache

Every run gets a shared cache on each host it runs on. When any task on a host downloads or produces a file, that file lands in the host’s cache — and every later task of the run placed on that host gets it symlinked in instantly, with zero network transfer and zero scratch disk, instead of re-downloading it from your bucket. The scheduler routes tasks toward hosts that already hold their inputs. On a cache miss it just fetches from S3 as normal — correctness never depends on the cache.

The per-host cache size is controlled by cacheBudgetGib (default 200). It’s a quota on host-side storage, not a reservation on your workers — it doesn’t add to worker disk, isn’t billed as worker disk, and doesn’t reduce how many workers fit. When a cache fills, the least-recently-used entries are evicted to make room; evicted files simply re-fetch from S3 on next use.

Size disk for outputs, not inputs

Cached inputs arrive as symlinks that consume no scratch quota, so a task’s disk directive only needs to cover what the task writes — outputs, temp files, logs. Pipelines that sized disk to hold copies of large inputs can shrink those directives substantially.

Set your cpus deliberately

Previously, tasks whose inputs overflowed scratch would fail with ENOSPC and retry at escalated resources (cpus/disk × attempt). With inputs now costing zero scratch, those tasks succeed on the first attempt at their declared resources. If your pipeline was implicitly relying on retry escalation for speed (common with stock nf-core configs on large data), declare the CPU counts you actually want — e.g. cpus 24 on alignment processes.

Disk is grow-only on reuse — a worker’s disk grows to the high-water mark of what its tasks need and doesn’t shrink, so a task with a dynamic disk { 50.GB * task.attempt } directive escalates naturally on retry.

The worker pool

• poolMax (default 64) caps how many workers a single run requests at once.
• idleTimeout (default 5min) controls how long an idle worker stays warm before it’s reclaimed. Between tasks a worker shrinks to a small idle floor; if no new task arrives within idleTimeout, it’s reaped.

Keep the pool warm

For pipelines with bursty phases (a gap between stages, then a big fan-out), raise idleTimeout so workers survive the gap and later stages reuse them — reused workers skip cold-start and keep their cached data.

When the fleet is full

A run’s pool spreads across the fleet, placed with locality in mind (related tasks cluster so their data stays local). When capacity is fully committed and the pool tries to grow, the executor doesn’t fail — it waits for a slot: tasks defer and retry until a worker frees up. In the run log you’ll see:

[ccloud] can't grow — task <name> waiting for a slot

This is normal backpressure, not an error — the pipeline still completes, just bounded by available capacity. If you see a lot of it, your run is larger than the fleet can hold at once; smaller workers (lower disk directives / defaultDiskGib) let more fit.

Some accounts also carry an aggregate compute cap — a limit on the total vCPUs and RAM your busy workers can hold at once. Hitting it behaves exactly like the fleet being full: new tasks wait and dispatch as your running tasks finish, the way a batch queue holds jobs at its max-vCPUs limit. It’s backpressure, not a failure — the run completes, just bounded by the cap.

There’s also a hard per-account ceiling on concurrent workers; a request past it is rejected until some free up.