Last week in scylladb.git master (issue #186; 2023-07-02)

This short report brings to light some interesting commits to scylladb.git master from the last week. Commits in the be5b61b870…1ab2bb69b8 range are covered.

There were 96 non-merge commits from 18 authors in that period. Some notable commits:

The row cache holds frequently-read rows. When a row is written with the TTL (time-to-live) option, it is set to be automatically deleted after a certain time. If it’s in the cache, however, it will continue to occupy memory, reducing cache utilization. This is now improved, as the cache will detect and remove expired rows when they are read. Infrequently read rows will be removed from the cache using the least-recently-used mechanism.

It is now possible to use TLS certificates to authenticate and authorize a user to ScyllaDB. The system can be configured to derive the user role from the client certificate and derive the permissions the user has from that role.

After repair or data movement due to node additions or removals, materialized views need to be updated. This process involves reading from all sstables except those that have been streamed or repaired. This was slow, and is now optimized, speeding up repair and data movement on clusters that have materialized views.

A race condition between cleanup and regular compaction has been fixed.

ScyllaDB uses evictable readers in certain places to allow the system to cancel ongoing reads to reclaim memory, with the ability to resume those reads later. A bug in the mechanism caused an entire partition to be consumed unnecessarily, which could lead to out-of-memory errors. This is now fixed.

While using a lightweight transaction, if inconsistent constraints were given on the clustering key, ScyllaDB would crash. This is now fixed.

ScyllaDB optionally uses Raft to coordinate changes to the schema and topology. It now attempts to merge adjacent changes to reduce overhead.

A GROUP BY query ought to return one row per group, except when all rows of a group are filtered out. However, ScyllaDB returned a row even for fully-filtered groups. This is now fixed, and ScyllaDB will not emit rows for filtered groups.

ScyllaDB will now wait for all nodes to be healthy before attempting to bootstrap a new node.

When using Raft for topology and schema changes, ScyllaDB will force the schema and topology to be transferred to new nodes. Previously, ScyllaDB relied on other mechanisms to transfer the schema.

A rare stack overflow in some repair scenarios has been fixed.

See you in the next issue of last week in scylladb.git master!