Last 2 weeks in scylla-cluster-tests.git master (issue #46; 2024-05-10)

This short report brings to light some interesting commits to scylla-cluster-tests.git master from the last 2 weeks.
Commits in the fdef448a…fa9af3cf range are covered.

There were 44 non-merge commits from 10 Software Engineers in Test and 2 Software Engineers in that period. Some notable commits:

Longevity tests can now be executed on Jenkins using a docker backend, which allows for testing in AWS environments similar to local setups. These tests are run on sct-runner instances equipped with NVMe disks.

The cluster provisioning process was sped up by avoiding re-execution of the machine configuration script if it has already been run, reducing the risk of bugs related to syslog-ng reconfiguration.

A new check ensures that non-voter nodes do not remain in the cluster with consistent-topology changes.

Nodetool scrub in validate mode is now utilized to verify the Scylla cluster post-longevity test. In the event of corruption, affected sstables are quarantined and uploaded to S3, and a corresponding error event is logged. This feature was introduced in multidc tier1 tests and will be expanded to additional tests.

The monitoring instance now uses a prepared image (version 4.6.2) to expedite provisioning on AWS and GCE, accompanied by updated documentation on image updates.

The Latte loader image size was significantly reduced from 580MB to 35MB, and its version updated, now used to test custom workflows.

Tests now have the ability to reserve AWS instances for their duration, which helps mitigate capacity issues common in performance tests using placement groups. It also includes fallback options to another AZ to increase test run chances.

Artifact tests now include running scylla-doctor and validating the JSON output file.

See you in the next issue of last week in scylla-cluster-tests.git master!