Resources & scaling

Task sizing

Each task is dispatched to a worker sized to that task’s Nextflow resource directives. Declare them in your pipeline or a config as usual:

process STAR_ALIGN {
    cpus   12
    memory 36.GB
    disk   50.GB
}

When a task moves onto a worker, the executor resizes that worker to fit — in milliseconds, without recreating the container. A chain of stages with different sizes runs on the same warm worker, resized between each.

Defaults for undeclared directives

If a process doesn’t declare a directive, the executor fills it in:

Directive	Default	Set via
cpus	1	Nextflow default
memory	4 GiB	ccloud.defaultMemGib
disk (scratch)	20 GiB	ccloud.defaultDiskGib

Raise defaultMemGib / defaultDiskGib in the ccloud { } block if your pipeline’s undeclared tasks need more.

The locality cache

Each worker keeps a persistent local keep-folder cache across tasks. When a downstream task lands on the worker that already produced its inputs, those inputs are hard-linked in instantly instead of being re-downloaded from your bucket. On a cache miss it just fetches from S3 as normal — correctness never depends on the cache.

The cache size is controlled by cacheBudgetGib (default 20), which is reserved on each worker on top of its scratch disk.

Raise cacheBudgetGib for large-file pipelines

The cache only helps for files that fit within its budget. For WGS-scale work where intermediate BAMs run 100–150 GiB, a 20 GiB cache skips them entirely — set cacheBudgetGib = 200 (or more) so big intermediates get cached and reused.

The size trade-off

cacheBudgetGib is reserved up front on every worker, so a larger cache means fewer workers fit for a given amount of fleet disk. The default of 20 favors more parallelism; raise it deliberately when locality on large files matters more than pool width.

Disk is grow-only on reuse — a worker’s disk grows to the high-water mark of what its tasks need and doesn’t shrink, so a task with a dynamic disk { 50.GB * task.attempt } directive escalates naturally on retry.

The worker pool

• poolMax (default 64) caps how many workers a single run requests at once.
• idleTimeout (default 5min) controls how long an idle worker stays warm before it’s reclaimed. Between tasks a worker shrinks to a small idle floor; if no new task arrives within idleTimeout, it’s reaped.

Keep the pool warm

For pipelines with bursty phases (a gap between stages, then a big fan-out), raise idleTimeout so workers survive the gap and later stages reuse them — reused workers skip cold-start and keep their cached data.

When the fleet is full

A run’s pool spreads across the fleet, placed with locality in mind (related tasks cluster so their data stays local). When capacity is fully committed and the pool tries to grow, the executor doesn’t fail — it waits for a slot: tasks defer and retry until a worker frees up. In the run log you’ll see:

[ccloud] can't grow — task <name> waiting for a slot

This is normal backpressure, not an error — the pipeline still completes, just bounded by available capacity. If you see a lot of it, your run is larger than the fleet can hold at once; smaller workers (lower cacheBudgetGib / defaultDiskGib) let more fit.

Some accounts also carry an aggregate compute cap — a limit on the total vCPUs and RAM your busy workers can hold at once. Hitting it behaves exactly like the fleet being full: new tasks wait and dispatch as your running tasks finish, the way a batch queue holds jobs at its max-vCPUs limit. It’s backpressure, not a failure — the run completes, just bounded by the cap.

There’s also a hard per-account ceiling on concurrent workers; a request past it is rejected until some free up.