S3 Integration

Carolina Cloud integrates with S3-compatible storage in two ways: you can bring your own S3 credentials at instance creation, or provision a managed storage bucket through the dashboard.

Bring your own S3 credentials

When creating a container, you can pass S3 credentials that will be pre-configured on the instance. This is useful for pulling reference data, uploading results, or working with existing data lakes.

Supported flavors: base, genomics, marimo, rstudioserver

Via the API:

curl -X POST https://console.carolinacloud.io/api/instance/ \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "resource_type": "container",
    "flavor": "rstudioserver",
    "n_vcpus": 8,
    "mem_gib": 32,
    "disk_size_gib": 500,
    "aws_access_key_id": "AKIAIOSFODNN7EXAMPLE",
    "aws_secret_access_key": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY",
    "aws_endpoint_url": "https://s3.wasabisys.com",
    "aws_prefill_bucket": "my-bucket/reference-genomes/"
  }'

When aws_prefill_bucket is specified, the contents of that bucket path are downloaded into ~/s3-prefill/ on the instance at startup. The aws CLI is pre-configured so you can run aws s3 ls immediately.

Via the dashboard:

Expand the S3 Integration section in the creation modal and fill in your credentials. If you have Carolina Cloud storage set up, click Use my Carolina Cloud bucket to auto-fill.

Carolina Cloud managed storage

Carolina Cloud can provision a Wasabi S3 bucket for you directly from the dashboard. This gives you a persistent storage layer that survives instance deletion, useful for keeping data between ephemeral compute sessions.

Provision a bucket:

From the Storage section of the dashboard, click Provision Storage. You will receive:

• An access key and secret key
• A bucket name
• The Wasabi endpoint URL

These credentials can be auto-filled into any new Marimo or RStudio Server instance.

Browse your bucket:

The dashboard includes a built-in file browser for your Carolina Cloud bucket. You can view folders and files without leaving the console.

Delete a bucket:

Deleting your bucket removes all objects, the bucket itself, and the associated IAM user. This action is permanent.

Automatic bucket mount

If you have Carolina Cloud storage provisioned, your bucket is automatically mounted at ~/ccloud-s3/ on every instance you create. There is nothing to configure — we handle the credentials and mount for you.

This uses s3fs-fuse under the hood, which means your S3 bucket behaves like a regular folder. You can ls, cp, cat, and reference files by path from any tool — Python, R, the terminal, whatever you’re working in:

# Browse your bucket
ls ~/ccloud-s3/

# Copy results into your bucket
cp results.csv ~/ccloud-s3/experiments/

# Read directly from Python, R, etc.
# pd.read_csv("~/ccloud-s3/datasets/sample.csv")

Any file you write to ~/ccloud-s3/ is persisted in your Wasabi bucket immediately. Any file already in your bucket is readable the moment your instance starts. Your data follows you across every instance you create — spin one up, do your work, tear it down, and the data is still there next time.

When to use what

• Bring your own credentials when you already have data in S3/Wasabi/MinIO and want your instance to access it at startup.
• Carolina Cloud storage when you want a simple, managed place to persist data between compute sessions without managing your own S3 infrastructure.