Quickstart — with a head container

This walks you from nothing to a running nf-core pipeline on the ccloud executor, driven from a Carolina Cloud head container.

Two ways to drive a run

This page uses a head container — a pre-wired driver we host, with your executor token and pipeline-bucket credentials already baked in. Nothing to install.

Prefer to drive from your own machine? The nf-ccloud plugin is in the Nextflow plugin registry, so you can run from a laptop, HPC login node, or CI runner with nothing but Nextflow and a token — see Quickstart — from your own machine.

1. Create a Nextflow head

In the console, choose Create → Nextflow, give it a name, and create it. That’s the whole form — the head is a fixed small size (it only runs the Nextflow driver, not your tasks), so there’s nothing to size. It lands on whatever host has capacity.

The head comes pre-loaded with everything the driver needs:

• Nextflow and the nf-ccloud executor plugin,
• a scoped access token injected as CCLOUD_API_KEY (no API key to paste),
• the Carolina Cloud endpoint injected as CCLOUD_ENDPOINT,
• a starter nextflow.config.

If you don’t already have a pipeline bucket, one is created for you as part of making the head. Its credentials take up to a minute to become active — so if your very first run fails with a permissions error, give it a moment and run it again.

2. Open the head’s terminal

Open the head’s web terminal (or SSH in). You’ll drive Nextflow from here.

3. Your storage is already wired up

There’s nothing to configure. The head is created with your pipeline bucket set as the work directory and its credentials injected as environment variables, so the global config it ships with already reads:

workDir = 's3://<your-pipeline-bucket>/work'

aws {
    accessKey = System.getenv('AWS_ACCESS_KEY_ID')
    secretKey = System.getenv('AWS_SECRET_ACCESS_KEY')
    client { endpoint = System.getenv('AWS_ENDPOINT_URL') }
}

The credentials are read from the environment rather than written into the file, so they never sit in a readable config. If you brought your own bucket when you created the head, the same applies with your credentials.

To send a run’s work somewhere else, override workDir in your own config — Nextflow merges yours over the head’s:

workDir = 's3://<your-pipeline-bucket>/experiment-2/work'

One constraint worth knowing up front: Nextflow talks to a single S3 endpoint per run, so your workDir, --outdir, and input data all have to sit on the same endpoint — same provider and region. They can be different buckets on that endpoint, as long as your credentials reach all of them. The credentials we mint for a Carolina Cloud pipeline bucket are scoped to that one bucket, so if you’re using ours, keep everything inside it.

Public reference data is the exception — pipelines pulling from public AWS (e.g. iGenomes) work regardless, because the executor fetches those anonymously for you. See Storage & buckets and Configuration.

4. Run a pipeline

NXF_SYNTAX_PARSER=v1 nextflow run nf-core/rnaseq -r 3.22.2 \
  -profile test \
  -c nextflow.config \
  -plugins nf-ccloud@1.0.0 \
  --outdir s3://my-bucket/results/

Two flags you can’t skip with nf-core pipelines

• -plugins nf-ccloud@1.0.0 — nf-core pipelines ship their own plugins { } block that overrides the one in your config. Without this flag on the command line, the run fails with Unknown executor name: ccloud.
• NXF_SYNTAX_PARSER=v1 — many released pipeline versions (e.g. rnaseq 3.22.2, sarek 3.4.4) use older config syntax that Nextflow 26’s strict parser rejects. This env var selects the legacy parser without changing the runtime. It’s per-pipeline — try a recent tag without it first, and add it only if the run fails to parse.

5. Watch it run

As the DAG opens, you’ll see the executor allocate pool workers, run tasks, resize workers between stages, and tear everything down when the pipeline finishes. Results are published to your --outdir; intermediate files stay in workDir (safe to delete after a successful run).

executor >  ccloud
[62/8a1f0c] NFCORE_RNASEQ:...:FASTQC (SAMPLE_1)     | 5 of 5 ✔
[a0/65dfb2] NFCORE_RNASEQ:...:TRIMGALORE (SAMPLE_1) | 5 of 5 ✔
...
Pipeline completed successfully

6. Stopping a run

To stop a pipeline early, press Ctrl+C in the head’s terminal. Nextflow cancels the running tasks and tears down your pool workers, and the run closes in the console within a few seconds. Pressing Ctrl+C more than once to force the exit is fine too — it still closes cleanly and just as quickly.

The only slower case is when the head goes away with no chance to shut itself down — you force-delete the head container mid-run, or it crashes. Your run and its workers still get cleaned up: the platform notices the head has gone quiet and closes everything out for you. That safety net just takes a few minutes to catch up, so the console keeps showing the run as active for a short while before it flips to stopped.

Rule of thumb: Ctrl+C stops a run cleanly and quickly. If the head is killed some other way — deleted or crashed — it’s still cleaned up automatically; just give it a few minutes to settle.

Next

• Watching a run — the runs list, live DAG, task views, and work-dir browser, and how each maps to your terminal and logs.
• Configuration — the full ccloud { } block and run flags.
• Resources & scaling — control worker size, the locality cache, and pool limits.
• Storage & buckets — references, multiple buckets, and what’s supported.