This short report brings to light some interesting commits to scylladb.git master from the last three weeks. Commits in the 092a88c9b9..7d562c24b1 range are covered.
There were 198 non-merge commits from 31 authors in that period. Some notable commits:
When loading sstables into the database with nodetool, there is now a –skip-cleanup option when the user wishes to defer cleanup to a later time. This allows a single cleanup operation to be run for many sstable loads.
Materialized view updates perform a read-modify-write operation on the base table. To prevent overload, we queue some of the reads. Previously, this queue had a limit of some number of entries beyond which reads would be rejected. This could cause base/view inconsistencies. The queue limit is now removed and we rely on throttling the base table writes to control its length.
Some alternator tests were migrated from dtest to test.py. This marks the beginning of an effort to migrate all dtests into scylladb.git.
We now manage temporary memory for zstd inter-node compression ourselves, in order to reduce allocation stalls.
The compaction history table now has additional columns for statistics.
There is now a custom index class (useful in CREATE INDEX) for vector indexes. Note it is not yet possible to query using the new index.
The native CQL transport now supports the metadata ID extension (from protocol version 5) that allows updating row metadata for prepared statements for SELECT * queries (when columns were added or removed) or SELECT udt queries (when the user define type definition changed). Note a driver that supports the extension is required to make use of this.
The nodetool refresh command now supports the –scope option, allowing data to be streamed only to the local node, local rack or local datacenter. This is useful when restoring from a backup that contains individual sstable sets for each rack.
A bug which could cause data resurrection in materialized views (but not in the base tables) due to mixing up purge times for regular tombstones and shadowable tombstones was fixed.
The process for merging tablets (which happens when a table’s size decreases) did not update the row cache about the merged sstables, causing some data to be missed during full scans. This is now fixed.
See you in the next issue of last week in scylladb.git master!