Watching a run
Every run shows up in the console the moment the head opens it. You can watch the graph fill in live, drill into any task, and read its logs and work-dir files without leaving the browser. This page walks through each part of the UI and shows how it lines up with what you see in the head’s terminal, the Nextflow control files, and your outputs.
Everything refreshes on its own — the runs list, the DAG, and the task panels poll about every 10 seconds, so the console tracks your terminal within a few seconds.
Finding your runs
Section titled “Finding your runs”In the left sidebar, click Nextflow (the diagram icon). That’s the runs list — every pipeline you’ve launched from any head. From a run you open the DAG page, the live graph for that single run.
The runs list
Section titled “The runs list”Each row is one run: its name (the Nextflow run name, or a short id), a status badge, when it started, how long it’s taken, and an Open DAG → link.
If you relaunch with -resume, the retries stack under one row — click N attempts to expand the earlier tries.
Status vs. outcome
Section titled “Status vs. outcome”The badge shows one of two things, and the distinction matters:
- Status is the run’s lifecycle on Carolina Cloud: active (running) → closed (the head shut the run down). aborted means the platform force-closed it — for example, after you deleted the head and the cleanup caught up (see Stopping a run).
- Outcome is Nextflow’s own verdict: Succeeded or Failed. Once Nextflow reports it, it replaces the lifecycle badge.
So a closed badge with no outcome means the run ended without Nextflow recording a result (you stopped it early). Succeeded / Failed is the pipeline’s real result.
The run drilldown
Section titled “The run drilldown”Click a row to expand it. This shows the run’s pipeline and profile, the Nextflow version, task counts, compute cost, and — most useful — the exact command line that launched it, with Copy command and Copy resume command buttons. This is the same header nf-core prints at the top of your terminal. Copy resume command adds -resume so you can relaunch and skip everything that already finished.
The live DAG
Section titled “The live DAG”Open DAG → opens the graph. It’s the same DAG Nextflow builds internally, rendered live. Each node is a process (like STAR_ALIGN), not an individual task — one node covers all the samples that run through that step.
The Guide button (top right) is your legend. Node colors:
| Color | State | Meaning |
|---|---|---|
| grey | waiting | not started yet |
| blue | running | executing now |
| amber | partial | some tasks done, some still going |
| teal | cached | resumed from cache (a -resume hit) |
| green | done | finished — every task succeeded |
| red | failed | a task’s latest attempt failed |
| dashed | skipped | a conditional branch that didn’t run |
Retries repaint the node live. If a task fails, the node goes red — even while sibling tasks in the process are green-lit. If the pipeline retries the task (errorStrategy 'retry'), the node turns blue again the moment the new attempt dispatches, and green once every task’s latest attempt has succeeded. A failure that was retried away leaves no red behind. A node that stays red means the newest attempt of some task in it failed — either its retries are exhausted, or the next attempt hasn’t dispatched yet.
How this maps to your terminal: the executor > ccloud (N) block in the head terminal lists each process with X of Y ✔. The DAG is the visual version of that block. A node going blue → green is the same process going from running to ✔; a teal node is the cached: N ✔ you see on a resume; a red node is the process the terminal’s ERROR ~ block names.
The header summary line — elapsed/took, processes done, running, done, errors — is the console’s version of the terminal’s live counters.
Click any node to open its task panel.
The task panel
Section titled “The task panel”Clicking a process node opens a terminal-style panel across the bottom. A Tasks tab lists every task in that process; each task you open gets its own tab, labelled by its short work-dir hash. Drag the top edge to resize the panel.
The task list shows each task’s name (e.g. STAR_ALIGN (SAMPLE_1)), its command, the attempt number, start/finish times, and a status pill:
- completed (green) · running (blue) · failed (red) · submitted (grey — queued, waiting for a worker) · cached (olive) · cancelled (grey)
These are per-task states — one task per sample or shard. The node color on the DAG rolls all of them up into a single per-process state.
Task detail — the debugging view
Section titled “Task detail — the debugging view”Click a task for its detail tab. This is where the console meets the Nextflow work dir.
Facts: status, exit code, attempt, the task’s compute cost, the Container (the worker it ran on), and the Work dir — the s3://… path Nextflow created for this task. That work dir is the [xx/xxxxxx] hash you see next to the task in the terminal.
Resource metrics:
| Metric | What it is |
|---|---|
| Allocated | the vCPU · RAM · disk the task’s worker was sized to |
| Runtime | wall-clock time the task ran |
| CPU | average vCPU actually used |
| Peak RAM | peak memory used |
Allocated is captured when the task is dispatched, so it’s there from the moment the task starts — including on a task that’s still running or one that died mid-flight. Each attempt shows its own allocation, so a retry with an escalating directive (memory { 8.GB * task.attempt }) shows what that attempt actually got. When a task fails, this row is the denominator to check: an OOM kill reads against the RAM it had, a no space left on device against its disk.
The usage numbers (Runtime through Peak RAM) are read from Nextflow’s .command.trace (realtime, %cpu, peak_rss), which only exists after the task completes — while it’s running they show —. Lined up against Allocated, they make it easy to spot a process that’s over- or under-provisioned.
The work-dir browser and control files
Section titled “The work-dir browser and control files”The task detail tab has a built-in, read-only browser for the task’s work dir — the exact directory Nextflow staged the task in.
Three pinned quick-views at the top open the files you reach for most:
- Script →
.command.sh— the exact command Nextflow ran for this task - Stdout →
.command.out— the task’s standard output - Stderr →
.command.err— the task’s standard error (where a tool’s failure message lands)
Below the pins, a folder/file list shows everything else in the work dir — your staged inputs, the outputs the task produced, and the rest of Nextflow’s control files. Click any file to preview it. Previews are format-aware: images (PNG, JPEG, SVG), PDFs, and HTML reports render inline, text files show as text, and anything else (BAMs, compressed archives) gets a download link.
These are the standard Nextflow control files, so what you see in the browser is exactly what you’d cat if you opened the work dir on the command line:
| File | What it is | In the UI |
|---|---|---|
.command.sh | the process command | Script pin |
.command.out | standard output | Stdout pin |
.command.err | standard error | Stderr pin |
.command.run | Nextflow’s task wrapper (input staging, environment setup) | in the file list |
.command.trace | the resource metrics | powers the metrics panel above |
.exitcode | the task’s exit status | the Exit fact |
.command.log | combined stdout + stderr | in the file list |
If the run used your own bucket, the panel shows the work dir’s s3:// path instead of listing files — we don’t store your storage credentials, so there’s nothing for the browser to read with. Carolina Cloud pipeline buckets browse normally.
When a run fails
Section titled “When a run fails”A failed run shows a red Failed badge and a Why? button on the DAG header.
Why? opens a summary: which process failed, the specific task, and the error message — the same ERROR ~ Error executing process > … block Nextflow prints in the terminal. View failed task → jumps straight to that task’s detail tab, so you go from “the run failed” to the failing task’s .command.err in two clicks.
The header’s errors counter is also a drilldown: whenever it’s non-zero, click it for a dropdown of every failed task in the run — including earlier attempts that were later retried away. Each row shows the task, its exit code, and the attempt number; clicking one opens that task’s detail tab directly.
Debug log for LLM
Section titled “Debug log for LLM”Next to Why? (and on the run’s row in the runs list) there’s a Debug log for LLM button. One click builds a markdown bug report for the failed run and copies it to your clipboard, ready to paste into Claude, ChatGPT, or whatever assistant you debug with — no screenshotting the DAG or hand-copying stack traces.
The report bundles everything an assistant needs to reason about the failure:
- The launch command line, pipeline and revision, Nextflow version, outcome, and run duration
- The failed task: exit code, attempt number, requested CPU / RAM / disk against the peak usage it actually hit — with a flag when the numbers point to an OOM kill — plus its work dir, error strategy, and the history of earlier attempts
- The sizes of the staged input files, which catches the “one sample is 10× the others” class of failure at a glance
- Head-and-tail excerpts of the task’s control files:
.command.err,.command.sh,.command.out,.command.log,.command.trace, and.exitcode - Your pipeline params, and Nextflow’s own error message and report
- A short primer on how Carolina Cloud runs Nextflow, so the assistant understands the platform instead of guessing
The report is built fresh when you click (the button shows Building… for a moment while we read the control files), and it never includes credentials or bucket keys. If the run used your own bucket rather than Carolina Cloud storage, the control-file excerpts are omitted — we can’t read your bucket — but the run, task, and error details are still included.
Browsing your outputs
Section titled “Browsing your outputs”If the run was launched with an s3:// --outdir, the DAG header shows a Browse outputs button. It opens the bottom panel on your output directory — the same folder/file browser and previews as the work-dir view, so a MultiQC HTML report, a PDF, or a plot renders right there.
Nextflow publishes results as processes finish, so the outputs panel fills in as the run progresses — an empty listing just means nothing has been published yet. You don’t have to wait for the run to complete to peek.
If --outdir points at your own bucket (not Carolina Cloud storage), the panel shows the s3:// path to open with your own S3 client instead — we don’t store your credentials.
One-click Open in JupyterLab / Open in RStudio buttons — an analysis session with your outputs pre-mounted — are coming soon; they’re visible in the panel but not live yet.
The DAG header and the run drilldown show Compute cost — the pool-worker compute for the run. It excludes storage, and each task’s own cost is on its detail tab.