This short report brings to light some interesting commits to scylladb.git master from the last week. Commits in the ab32ce6b45…6d8e9645ce range are covered.
There were 185 non-merge commits from 23 authors in that period. Some notable commits:
ScyllaDB will now tune the number of allowed open files descriptors for very large nodes.
When replaying hints for a node, we now consider whether the node is leaving the cluster. If it is, we send the hints to all replicas to avoid losing the data when the node leaves.
Major compaction now supports a new option to only check existing sstables during tombstone garbage collection; this can increase the effectiveness of garbage collection for partitions that are updated frequently.
Scrub, split, and upgrade compactions increase the compaction group shares in order to guarantee progress; however this is unnecessary as they now run in the maintenance group. The share increase is therefore removed.
Commitlog is now able to store entries larger than half a commitlog segment. This limitation caused problems with large clusters, as cluster metadata could exceed this limit. Large entries are now fragmented and split over multiple segments.
A bug in computing whether to flush all memtables was fixed.
The system_distributed.view_build_status was moved to the system keyspace and is now managed by Raft.
Scrub/validate compactions will now verify checksums for uncompressed sstables.
The jmx submodule was removed from the source tree. With nodetool now talking directly to the REST API, it is no longer necessary. JMX is still available as a separate package.
The heuristics for purging tombstones during compaction were improved, leading to less tombstone accumulation.
See you in the next issue of last week in scylladb.git master!