Limitations of tiny partitions

ScyllaDB
, , , ,
1

I am having around 150 million rows in a table. Each row is a partition which means 150 million partitions. Size of each partition is 24kb.

I know that small partitions work perfectly for even distribution of data without hotspots. But, having partitions as small as 24kb can be a problem in managing them in cache (c++ objects limitations) or using mv’s/secondary index for this table in scylla.

Is this true? Is this improved in latest version of scylla?

2

No problem. ScyllaDB is designed to support billions of partitions.

3

Thanks for your reply.

Just to cross check, tiny partitions won’t be blocker for

  • materialized view’s and secondary indexes during repairs.
  • caching tiny partitions in memory

Thanks.

4

You should be fine. Plus, if needed, you can always tune bloom filter, repair intensities, and compaction settings in case you find anything weird later on.

1 Like