S3 backed storage experimentation

ScyllaDB
, ,
1

Installation details
#ScyllaDB version: 2025.3.3
#Cluster size: 1 node
os (RHEL/CentOS/Ubuntu/AWS AMI): Ubuntu

Hello,

I wanted to test out the experimental S3 backed storage for keyspaces but I am a bit confused about the configuration part.

I don’t have AWS and I am not at all familiar with how it works (which may explain the confusion).

I have my own S3 provider with a custom domain. My buckets are available through this provider using ONLY virtual-hosted-style URLs e.g https://<bucket>.<domain>/.

Does Scylla support this ? I don’t see any bucket field in the object_storage_endpoints option in scylla.yaml.

2

The bucket is defined on the keyspace definition, something like

CREATE KEYSPACE ks
  WITH REPLICATION = {
   'class' : 'NetworkTopologyStrategy',
   'replication_factor' : 1
  }
  AND STORAGE = {
   'type' : 'S3',
   'endpoint' : 's3.us-east-2.amazonaws.com',
   'bucket' : 'bucket-for-testing'
  };

So, once you define your endpoint in the scylla.yaml you can create a keyspace which will be using this endpoint. As of the endpoint you mentioned you can try and do the following

CREATE KEYSPACE ks
  WITH REPLICATION = {
   'class' : 'NetworkTopologyStrategy',
   'replication_factor' : 1
  }
  AND STORAGE = {
   'type' : 'S3',
   'endpoint' : 'mybucket.domain.com',
   'bucket' : '/'
  };

Try to leave the bucket empty. If it doesnt work for you and you can tolerate additional level of directories created try to do the following

CREATE KEYSPACE ks
  WITH REPLICATION = {
   'class' : 'NetworkTopologyStrategy',
   'replication_factor' : 1
  }
  AND STORAGE = {
   'type' : 'S3',
   'endpoint' : 'mybucket.domain.com',
   'bucket' : 'mybucket'
  };

For the reference on how to configure object storage

1 Like