[RELEASE] Scylla Manager 3.4.0

Release Notes
, ,
1

The ScyllaDB team announces the release of ScyllaDB Manager 3.4.0, a production-ready patch release of the stable 3.4 branch.

This release focuses on controlling the restore procedure and improving its performance.

ScyllaDB Manager is a centralized cluster administration and recurrent tasks automation tool.

ScyllaDB Manager is available for ScyllaDB Enterprise customers and ScyllaDB Open Source users. With ScyllaDB Open Source, ScyllaDB Manager is limited to 5 nodes. See the ScyllaDB Manager Proprietary Software License Agreement for details.

In version 3.4.0, we reimplemented the core of the restore procedure in order to improve task parallelisation, shard utilization, and overall performance.

Chart1600×640 63.2 KB

The chart above compares the restore of identical data on clusters of the same configuration. Scylla Manager 3.4 improved overall CPU utilization during the process, resulting in a shorter time required for a full restore.

Chart1600×649 59.8 KB

This second chart shows the results of a restore performed on a 9-node cluster against another 9-node cluster. Once again, Scylla Manager 3.4 improves overall resource utilization during the process. In previous versions of Scylla Manager, a high batch size of 40 improved performance on a 3-node cluster but led to degraded CPU usage during restores on a 9-node cluster. However, with Scylla Manager 3.4, this is no longer an issue, as a higher batch size leads to better resource utilization, ultimately reducing restore time.

We also added new restore task flags that will make it easier to configure restore to run as fast as possible:

  • ‘–transfers’, ‘–rate-limit’, and ‘–unpin-agent-cpu’ allowing to control ScyllaDB Manager Agent’s download configuration
  • ‘–allow-compaction’ allowing to control auto compaction for the time of the restore procedure

Please see ScyllaDB Manager restore documentation for flag details and recommendations on restore speed.

Moreover, we also increased restore performance observability by introducing new metrics that can be used for calculating download and load&stream bandwidths:

  • ‘scylla_manager_restore_downloaded_bytes’
  • ‘scylla_manager_restore_download_duration’
  • ‘scylla_manager_restore_streamed_bytes’
  • ‘scylla_manager_restore_stream_duration’

We also added the per shard download and load&stream bandwidths to the output of the sctool progress command.

Apart from restore oriented changes, the 3.4.0 release also contains a few important bug fixes:

  • #4037 - fixed a bug where interrupted restore of a backup containing Materialized Views couldn’t be resumed
  • #3922 - fixed a bug where it was possible to make an incomplete backup of a keyspace with ‘SimpleStrategy’ replication strategy when backing up only a subset of data centers

ScyllaDB Enterprise customers are encouraged to upgrade to ScyllaDB Manager 3.4.0 in coordination with the ScyllaDB support team.

The new release includes upgrades of both ScyllaDB Manager Server and Agent.

Useful Links:

Scylla Manager 3.4.0 supports the following Scylla Enterprise releases:

  • 2024.2
  • 2024.1
  • 2023.1

And the following Open Source release (limited to 5 nodes see license 1):

  • 6.1
  • 6.0

You can install and run Scylla Manager on Kubernetes using Scylla Operator. More here.

1 Like