Migrating to ScyllaDB from Cassandra MongoDB DynamoDB

ScyllaDB
1

What does the typical migration process from (Cassandra/DynamoDB/Mongo) look like? How long does it take?

3

This is a very wide question (you can learn more in a Scylla University dedicated lesson).

You need to consider whether you’ll be doing a Hot/Live or a Cold/Offline migration.
There’s the live traffic aspect and the historical data aspect.

If you’ll be doing a HOT migration, here is the sequence you should follow:

  1. Deploy the destination cluster (ScyllaDB, with the proper schema/data modeling)
  2. Live traffic: Enable dual writes
  3. Historical Data forklift/migration
  4. Verification period: dual reads (source is still the source of truth)
  5. Cutover to the destination cluster (as the source of truth

Now let’s talk about historical data migration.

The following tools can be used to migrate historical data from Cassandra:

  • sstableloader (note to use the sstableloader that comes with Scylla, not the Cassandra one)
  • CQLSH COPY TO / FROM (export to csv, mostly up to 2M rows)
  • Similar tool to COPY FROM / TO is dsbulk, which has some optimizations that COPY TO / FROM doesn’t have
  • Copy over sstable files and using nodetool refresh
  • Scylla Migrator (Spark based) - full scan/cql reads on the source and cql writes to the destination.

The following tools can be used to migrate historical data from DynamoDB to ScyllaDB (CQL API / DynamoDB compatible API, a.k.a Alternator):

In order to migrate from MongoDB you’ll 1st need to design your schema / data model. A nice tool that can help you with that is Hackolade

As for data migration tools:

  • Export data to CSV and import to Scylla
  • Scylla Migrator

Happy Migration!

4

Thanks @Tomersan! Very helpful.
I’ll add that if you’re doing a migration, get in touch with us, we’re happy to help.
Also, in the upcoming LIVE event, we’ll have a dedicated session on migrating from DynamoDB.

5

Hate to necro this thread but it’s been about a year - has any of the information above changed substantially? We’re considering moving from our Cass 3.11 cluster (12 nodes, 2 DCs) to Scylla 5.1. Any recommendations or tips are welcome.

6

No substantial change that I’m aware of.
Later today, we’ll host another LIVE training event, and there will be a specific session on migrating from DynamoDB to ScyllaDB, including a hands-on example. Hope to see you there.

7

@Guy Can I see recordings of such events? If not, think about it, it might be useful for newcomers.

8

These events are available live only.
We’re working on updating the Migration lesson on ScyllaDB University.
Also, stay tuned for the next training event (we have one every few weeks) as well as for the upcoming ScyllaDB Summit.