This short report brings to light some interesting commits to scylladb.git master from the last week. Commits in the a93fd2b162…decbc841b7 range are covered.
There were 144 non-merge commits from 19 authors in that period. Some notable commits:
There is now a REST API for configuring Prometheus metrics label rewriting.
The hints synchronization point API allows an external user to wait for hints replay. Misuse of the API cookie could lead to unbounded memory usage; the cookie is now protected with a checksum.
ScyllaDB uses feature flags to coordinate rolling upgrades; a feature isn’t enabled until all nodes report they support that particular feature flag. Occasionally some older feature flags are considered “always on” and aren’t negotiated. A problem with non-negotiated features and storing feature flags in Raft group 0 would have prevented upgrades, but it was fixed.
Among its tasks, gossiper disseminates cluster state updates within the node. Node removal notifications were processed in the background, which could cause them to be reordered with other notifications, causing problems. This is fixed by moving processing to the foreground.
A recent regression when using GROUP BY together with the ttl() and writetime() pseduo-functions was fixed.
There is a new SELECT statement variant that allows seeing where the data that composes a selection comes from. Normally, cache, sstable, and memtable data are merged before output, but with this variant one can see the original source of the data. This is intended for forensics and is not a stable API.
A Seastar update will reserve more memory for the operating system in situations that previously led to out-of-memory errors. These situations are ARM machines with 64kB pages (rather than the usual 4kB), and transparent hugepages enabled. As a side effect ScyllaDB will run with less memory.
The compaction manager sometimes generates sstables composed of only tombstones, in order to safeguard against a crash causing data resurrection. If there isn’t a crash, these sstables can be safely deleted. However, they are sometimes picked up for compaction before they are deleted, wasting CPU cycles. They are now excluded from compaction.
The cassandra-stress tool now supports the Java driver’s rack-aware policy. This can reduce cloud inter availability zone networking costs, with the downside of less even load balancing if care isn’t taken to balance the application.
The CQL grammar incorrectly accepted nonsensical empty limit clauses such as SELECT * FROM tab LIMIT;. The errors were discovered later in processing, but with unhelpful error messages. They are now rejected.
The CQL grammar incorrectly accepted nonsensical INSERT JSON statements such as INSERT INTO tab JSON;, causing a crash. This is now fixed.
The mutation compactor now validates its input stream rather than the output stream.
A mistake in function type inference, which could lead the CQL statements to claim there is ambiguity when in fact there is none, was fixed.
See you in the next issue of last week in scylladb.git master!