I want to track the tombstone count in each sstable to determine when to execute compaction. Therefore, I added a new parameter, ‘tombstone_count,’ to ‘stats_metadata.’ The newly generated sstable statistics files with this parameter have no issues. However, when reading the older sstable statistics files, the value of this parameter appears as a random value. I would like to know if this modification could cause any problems.
I traced the process of reading
statistics , and it seems that when parsing this parameter, it directly reads the content of size uint64_t and assigns this content to the value. However, since the files generated before the modification do not have this parameter value, it is like reading some extra content. Will this have any impact? After executing compaction to regenerate the sstable, this parameter value is normal.
You need to initialize this new member to 0. Having a random value when not present makes it useless.
Note that this count will not be correct when there are TTL’d cells. Also, a tombstone can only be purged under certain conditions (usually when it is 10 days old), which this count won’t be able to account for.
It is possible to calculate the exact amount of purgeable tombstone, with a script like this. Run this as: scylla script --script-file=/path/to/purgeable.lua /path/to/sstable.
Taking a step back, why do you want to start compactions by hand? Why not enable tombstone compaction instead, so ScyllaDB itself will watch sstables for purgeable tombstone ratio and will kick off compaction when this is above the configured threshold.
Thank you. I tried setting this parameter a default value of 0, but it still reads a random value when accessed.
Could you please advise on how to enable tombstone compaction for ScyllaDB to monitor the droppable tombstone ratio, and how to configure the threshold for this?
See Compaction | ScyllaDB Docs, in particular the options: enabled (enables tombstone compaction), tombstone_threshold and tombstone_compaction_interval.
I have got it, thank you.