The ScyllaDB team is pleased to announce the release of ScyllaDB Monitoring Stack 4.17.0
Related Links
The main changes in this release are a single configuration file for the monitoring stack, Grafana Alloy as the log collector, and rack grouping in the dashboards. The release also adds support for ScyllaDB 2026.4.
Version updates for ScyllaDB Monitoring Stack 4.17.0
-
Prometheus upgraded to version 3.16.0
-
Grafana upgraded to version 13.3.1
-
ScyllaDB data source plugin upgraded to version 0.8.0
Configure the stack from one file
-
The monitoring stack can now run from one YAML file instead of start-all.sh flags. The file holds the targets, alert rules, the Alertmanager configuration, Grafana settings, storage, and TLS. start-all.sh with flags keeps working as before. #2946
-
scylla_monitoring_config.py validates the file and applies it. It can check the file as a dry run, apply it once, or watch it and apply every change, for example as a systemd service. A change runs only what it needs: new targets need nothing, new alert rules need a reload, and a new storage path or version restarts the stack. #2946
-
The watcher checks before it applies and recovers when something is down. It checks prometheus.yml and the Docker Compose definition before applying them, restarts the stack when a component is down, and can return to the last file that worked. #2961 #2963
-
Both start-all.sh and compose mode are supported. #2970
Dashboards updates
All dashboards
- Rack grouping and a rack filter. The “by” selector has a Rack option, and there is a rack filter next to the DC filter, on the Overview, Detailed, Advanced, CQL, OS, Keyspace and Alternator dashboards. Racks matter more with tablets, because the load balancer equalizes each rack independently. Rack groups by DC and rack, since rack names often repeat across DCs. #2913
-
Rare events show their peak rate. Panels that count rare events, such as failures, timeouts and blocked requests, drew a single event as a low, wide plateau, and could lose it when zoomed out. They now show the peak rate, so a single event has the same height at every zoom level. #2923
-
One annotation list for all dashboards. The annotations that are rarely used are off by default and are in the dashboard controls menu, which keeps the top bar short. #2932
Detailed Dashboard
- The tablets panels have their own section. It adds per-rack tablets load panels and a Scheduler section with the load balancer’s call rate, the tablet transitions it schedules (migrations, repairs, rebuilds) and its resize decisions. Repairs and rebuilds are reported from ScyllaDB 2026.3. #2914
- Tablet Repair and Vnode Repair rows. The dashboards had no ScyllaDB-side repair panels, so tablet repair was not visible. A cluster uses either tablets or vnodes, so only the matching row is shown. Tablet Repair shows tablets repaired and failed per second, bytes sent, incremental repair bytes read and skipped, and the time spent on repair. Vnode Repair shows rows sent and received, bytes sent, and bytes read from disk. #2927 #2940
- The rate of failed CDC operations is shown. The Failed CDC operations panel divided the rate by the LWT read count, so its values were wrong. It now shows the rate itself, like the CDC Operations panel next to it. #2931
Advanced Dashboard
- A Host CPU row. It shows the host CPU time of each shard’s core by mode: user, system, iowait, irq, softirq, steal, nice and idle. The panels filter and group like the rest of the dashboard, so a stalling shard can be compared with the steal or softirq time on its own core. #2897
- A reactor stall histogram row. A single long stall was almost invisible in the task quota violations panel. The new row shows the number of stalls longer than a selectable cutoff (50 to 400 ms), the P99 and P999 stall, the average stall, and the time spent in stalls. #2935
Overview Dashboard
- Cache reads and misses are shown instead of row hits. The panel counted cache row hits and misses, so a workload whose reads hit or miss at the partition level showed many reads but almost no cache activity. It now shows cache reads and reads with misses, in the same unit as the Reads panel; the gap between the two lines is the cache hits. The panel is renamed Cache Reads/Misses. #2928
Keyspace dashboard
- A switch to show all tables. The table selector lists only the 20 most active tables. The new All tables switch removes the limit, so every table of the keyspace can be selected. #2921

Alternator Dashboard
-
Filtered rows panels. Two panels show the rows that Alternator filter expressions read, matched and dropped, and the percentage of rows that were dropped. A high percentage means that requests read many rows that the filter then discards. The row stays hidden until ScyllaDB reports the number of rows read. #2924
-
Operation size panels, also per table. The panels show the average, P95 and P99 size of each Alternator operation, such as GetItem, PutItem and BatchWriteItem, with a tab per operation. They show whether requests carry unusually large items or batches. The section is hidden when the metric is absent. #2933 #2934
OS Dashboard
-
The disk panels show the disk device. The latency panels showed one line per device with only the node in the legend, and the IOPS and bandwidth panels summed all the devices of a node. Each line now names its device, so a slow or busy disk can be told apart from the others. #2942
-
A Graviton PMU row. It shows PMU counters such as IPC, memory stalls and L2 misses next to the reactor and kernel CPU panels. The metrics come from a PMU textfile collector, not from the standard node_exporter, so the row is hidden when they are absent. #2954
Vector Search Dashboard
-
Indexes are identified by keyspace and index name. Two indexes with the same name in different keyspaces are now shown separately. #2919
-
Request metrics are split by operation. The vector store now labels its request latency with the operation (ann, bm25, highlight, like), and the request panels mixed all of them together. The request rate and latency panels now show a line per operation, and a new operation selector filters them. The four highlight panels, which were always empty, are removed. #2930
Alerts updates
-
nonLsaMemoryHigh. Non-LSA memory, mostly SSTable metadata such as bloom filters, can grow until the node runs out of memory. The alert warns when it is above 30% of a node’s memory and raises an error above 40%. #2937
-
ManagerDown. A warning when Prometheus can’t reach Scylla Manager for 5 minutes. Setups without a Manager stay quiet. #2937
-
Automatic SSTable quarantine. A Loki alert when ScyllaDB quarantines an SSTable. #2950
Breaking Changes
Grafana Alloy replaces Promtail as the log collector #2907
Promtail is deprecated upstream and no longer updated. The stack now starts Grafana Alloy by default, also with Docker Compose. Alloy listens for syslog on port 1514, the same port Promtail used, so the rsyslog configuration on the ScyllaDB nodes does not change. To keep Promtail, start the stack with --log-collector promtail. The new --no-log-collector flag runs neither Loki nor a collector; --no-loki is deprecated, still works, and logs a warning.
Alerts that notify more often #2925 #2937
The Alertmanager inhibit rules were reversed: a firing lower severity muted the higher severity of the same alert, so the higher one never notified. The higher severity now mutes the lower one, and the rules also match on cluster and instance. Expect notifications for warn and error levels that were muted before. In addition, nodeCLErrors is now an error and is reported per DC, and NoCql (Scylla Manager can’t connect to a node over CQL) is now a warning. If you keep your own Alertmanager configuration, copy the new inhibit rules into it.
“Your Panels” rows removed #2932
The placeholder rows for user panels are removed from the Overview and Keyspace dashboards. If you added your own panels through that mechanism, move them to your own dashboard.
make-compose.sh is deprecated #2946
make_compose.py replaces make-compose.sh, which now prints a deprecation warning. It accepts every start-all.sh option, and start-all.sh --compose runs it.
Local Thanos query #2953
The query started by --local-thanos no longer uses cluster as a replica label, so a central Thanos query can use it as a leaf in distributed mode. --thanos keeps the label.
Operational Changes
-
Support for ScyllaDB 2026.4
-
ScyllaDB data source plugin 0.8.0: when Grafana runs inside the cluster’s network and the nodes advertise a public address, the plugin connects to the nodes on their internal address #2958 #2960
-
start-all.sh: --no-manager runs the stack without Scylla Manager dashboards and scrape jobs #2961
-
start-all.sh: --cluster-label sets a cluster label on the stack’s own metrics and as a Prometheus external label #2946 #2953
-
start-grafana.sh: every GF_* variable set in env.sh or the environment is passed to Grafana #2971
-
start-grafana.sh: PLUGINS_PREINSTALL_DISABLED stops Grafana from installing plugins from grafana.com when it starts #2968
-
The Grafana and CQL passwords are passed through the environment and are no longer on the command line of the containers #2946
-
start-all.sh: --limit, --volume and --param reach every container #2946
-
start-traefik.sh: --reload for new certificates, and --config-only #2946
-
kill-all.sh: --traefik also removes the traefik container; without the flag it keeps running #2951 #2952
-
package_metrics_for_support.sh: the v1.1 packager, with --days, --force-daily and --diagnostics #2956
-
Machine images: Grafana’s files are read from the disk on boot, so its first start is shorter #2976
-
Machine images: only the Grafana plugins monitoring uses are shipped #2971
-
Machine images: Docker’s networks stay inside 172.17.0.0/16 #2973
-
Machine images: the ScyllaDB data source binary is installed on arm64 images #2943
Bug Fixes
-
The RPC panels on the Advanced dashboard showed no data unless exactly one domain was selected #2926
-
Rare RPC and CQL errors disappeared when zoomed out #2926
-
The disk and partition tables showed duplicate rows from different nodes #2929
-
Average latency and batch size panels gave wrong values with an aggregation function other than sum; they now use weighted averages #2931 #2934
-
Latency by shard showed nothing with a function other than sum #2938
-
Auto grid rows had a different height from grid panels #2941
-
Annotations lost their titles and tag keys #2955








