Quickstart — Nextflow API

The other two quickstarts have you open a terminal and type nextflow run. This one skips that entirely — one API call launches the pipeline, and you poll for status. No head to create, nothing to SSH into.

When to use this

• You’re building your own product or API on top of Carolina Cloud (reselling pipeline runs to your own customers).
• You’re automating from a script, a backend service, or CI — nowhere interactive to type a command.
• You don’t want to manage a driver at all: each launch gets its own ephemeral head, created for that one run and torn down automatically when it closes. Nothing persists, nothing to clean up yourself.

Prefer an interactive terminal? Use Quickstart — with a head container or Quickstart — from your own machine instead — this page is for programmatic launches.

1. Get an API key

Open API Key in the console and copy your key. Every call below sends it as Authorization: Bearer <your_api_key> — this is your full account key, not a scoped pipeline token (those are minted automatically per-launch, behind the scenes).

2. Point it at a bucket you own

outdir has to be an s3:// path in a Carolina Cloud bucket you own — your auto-provisioned pipeline bucket, a personal bucket, or an org bucket (org buckets: org admins only). See Storage & buckets for how bucket credentials work.

An external (non–Carolina Cloud) bucket can be write-probed at launch via external_aws_creds, but the run itself still publishes with your pipeline-bucket credentials, so an external outdir isn’t supported end-to-end yet — use a Carolina Cloud bucket.

3. Launch

curl -sS -X POST https://console.carolinacloud.io/api/pipeline-launch/ \
  -H "Authorization: Bearer $CCLOUD_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "pipeline": "nf-core/demo",
    "pipeline_type": "ref",
    "outdir": "s3://my-bucket/pipeline-runs/demo-1/out",
    "engine_flags": ["-profile", "test"],
    "pipeline_params": []
  }'

Field	Required	What it is
pipeline	yes	An nf-core name, a git URL, or (with pipeline_type: "script") the raw text of a main.nf
pipeline_type	no (default ref)	ref for a name/URL, script to submit source directly
outdir	yes	s3:// destination for --outdir — see step 2
config	no	nextflow.config text, layered on top of the head’s own executor config
engine_flags	no	Nextflow flags, e.g. ["-resume", "-profile", "test"]
pipeline_params	no	Pipeline params, e.g. ["--input", "samplesheet.csv"]
external_aws_creds	no	{endpoint_url, access_key_id, secret_access_key} — validates an external outdir bucket at launch (see the caveat in step 2)

Don’t set these yourself

--outdir is added for you from the outdir field — don’t repeat it in pipeline_params. -plugins nf-ccloud@… is injected automatically, and config text travels in the config field — so -plugins, -c, and -config in engine_flags are all rejected with a 400. If you don’t pass -name, a readable run name is generated for you.

You’ll get a response back immediately:

{"id": "45d91a6b-64b9-4711-b58b-727ad0d98d90", "state": "starting", "head_uuid": "1c90bead-ffd1-43aa-a71f-30ae8e8c6387"}

state: "starting" just means the request was accepted — the head is booting and nextflow run is being dispatched in the background. This is asynchronous: the call returns in well under a second, long before the pipeline itself has finished (or even started).

4. Poll for status

curl -sS https://console.carolinacloud.io/api/pipeline-run/45d91a6b-64b9-4711-b58b-727ad0d98d90/ \
  -H "Authorization: Bearer $CCLOUD_API_KEY"

Two fields matter, and the distinction is the same one the console DAG viewer uses:

• status is the run’s lifecycle: active (still running) → closed (ended normally) or aborted (force-closed — e.g. it never got the chance to shut down cleanly).
• outcome is Nextflow’s own verdict: succeeded or failed, or null if the run ended before Nextflow reported one (a very early crash, or a manual cancel — see below).

GET /api/pipeline-run/ (no id) lists your 200 most recent runs, newest first, if you’d rather poll a list than track one id.

5. If it fails — the debug log

curl -sS https://console.carolinacloud.io/api/pipeline-run/45d91a6b-64b9-4711-b58b-727ad0d98d90/debug-log/ \
  -H "Authorization: Bearer $CCLOUD_API_KEY"

Returns {"markdown": "…"} — a bug report built for pasting straight into an LLM: the failing task, its .command.err/.command.sh, resource usage, and Nextflow’s own error report. Since a launch-API run’s head is deleted once the run closes, this endpoint also carries a “scavenged from the head” section for these runs specifically — the tail of .nextflow.log and the exact config/script that was run, pulled off the head moments before teardown. It’s the only place that information survives to.

6. Cancel a run

curl -sS -X DELETE https://console.carolinacloud.io/api/pipeline-run/45d91a6b-64b9-4711-b58b-727ad0d98d90/ \
  -H "Authorization: Bearer $CCLOUD_API_KEY"

Tears down the run’s pool workers and (for a launch-API run) its head, right away. It’s a hard stop, not a graceful one — Nextflow doesn’t get a chance to shut itself down first, so outcome stays null on a manually-cancelled run, same as an early crash.

Note

NXF_SYNTAX_PARSER=v1 (which some released nf-core versions need — see the caution box in Quickstart — with a head container) is set automatically on every launch-API run. Other environment variables can’t be passed through this API yet; if a pipeline needs one, use a head container or your own machine.

Next

• Watching a run — every launch-API run shows up in the console DAG viewer too, exactly like a head-driven one.
• Storage & buckets — bucket ownership, credentials, and the one-endpoint rule.
• Configuration — what’s baked into every run vs. what you supply.