This short report brings to light some interesting commits to scylladb.git master from the last two weeks. Commits in the 509fb96dcf2..dc4ad9bfdb9 range are covered.
There were 221 non-merge commits from 32 authors in that period. Some notable commits:
Vector search can use function style ORDER BY ANN(…) syntax, similar to full-text search bm25().
Performance of large bitset construction (used in Bloom filters) was improved.
The default speculative retry option can now be adjusted in configuration.
The nodetool scrub command now checks digests on all components by default.
Tablet migration is no longer serialized with tablet split.
An sstable on object storage can now be shared when migrated to a different shard of the same node instead of copied.
A deadlock on Raft group 0 shutdown was fixed.
The Prometheus metrics exporter can now export as protobuf in addition to plain text.
An aggregation query that encountered a partition with static columns only could emit incorrect results. This is now fixed.
The compaction backlog tracker measures the amount of compaction work needed for a tablet. It was disabled for tablets created after the node started, whether by migration or by split, causing the node to lose track of the amount of effort needed for compaction. This now now fixed.
The sstable sets holding a tablet that is split are now configured with the correct token range, to guide their data structure heuristics correctly.
Incremental Compaction Strategy generates tombstone-only sstables as redo logs in case of a crash, which are then removed. In some cases this removal failed when racing with a tablet merge. This is now fixed.
Table restore now accepts a prefix for the location of sstables in the tablet-aware API.
The manual tablet migration APIs now validate replication constraints.
Time Window Compaction Strategy corrected its filtering of sstables for reversed reads, no longer losing rows due to incorrect filtering.
The logstor key/value storage engine now trains the progile-guided optimization build.
Deserialization of canonical mutations, used for metadata and by the logstor storage engine, was optimized.
The CLIENT_OPTIONS CQL startup configuration key was renamed to DRIVER_CONFIG. This is used in the system.clients table.
CQL pages queries with many results to avoid head-of-line blocking and stalls. The coordinator now records the query plan in the resume cookie it sends to the client, so that paged queries can survive addition and removal of indexes. This paves the way to more complex query plans.
The logstor storage engine, used to key/value workloads, now has a compaction controller that adjusts compaction bandwidth with workload.
Alternator, ScyllaDB’s implementation of the DynamoDB API, now supports multi-component global secondary indexes (GSIs).
The Raft implementation now supports leases to improve read performance when clock uncertainty can be bounded.
The logstor storage engine can now use sparse files for test deployments.
See you in the next issue of last week in scylladb.git master!