This short report brings to light some interesting commits to scylladb.git master from the last week. Commits in the 5b4e688668…6e3b997e04 range are covered.
There were 111 non-merge commits from 18 authors in that period. Some notable commits:
In Alternator, ScyllaDB’s implementation of the DynamoDB API, we now yield when generating large JSON responses to avoid inducing latency for concurrent requests.
The tablet load balancer now uses randomization to select candidates for tablet replica migration to avoid tablets for a particular table clustering in some nodes or shards, which can cause CPU imbalance.
The primary replica algorithm for tablets has been adjusted so that nodetool repair -pr balances repair work more evenly across the cluster.
Materialized view flow control is based on adjusting the write rate based on backlog. The backlog calculations are now more accurate.
The task manager provides observability to the operator about internal operations. It is now careful to conserve memory by not keeping state for completed tasks.
Hints were recently changed to be stored per host ID rather than per IP address. However, hints can still be stored with IP-based directories from older cluster. A bug draining those hints when a node leaves the cluster was fixed.
The bundled cqlsh version was updated to 6.0.20.
Repair and tablet migration are now serialized.
Schema-modifying statements (DDL) and authentication were moved to rely on Raft separately. A DDL statement that grants permissions (for example, CREATE TABLE) will now execute in a single transaction. This prevents failures from leaving only part of the operation committed.
The schema and topology coordinator now runs in the [gossip scheduling group](group0, topology coordinator: run group0 and the topology coordinator… · scylladb/scylladb@34cf5c8 · GitHub Author: Gleb Natapov gleb@scylladb.com); previously it was run in the streaming scheduling group. This prevents operations like repair and streaming from competing with metadata management.
A bug in write back-pressure management, that could trigger an out-of-memory condition, was fixed.
See you in the next issue of last week in scylladb.git master!