Hey everyone, I have a question.
So I have a deployment that includes ScyllaDB, that I do using helmfile (this is the helmfile). I already have Prometheus and Grafana in the deployment, because I’m using the kube-prometheus-stack helm chart. I was trying to add monitoring to ScyllaDB (from here), because IIUC ScyllaDB already logs some Prometheus metrics and if I enable the monitoring I can have access to the metrics and dashboards. Is that correct?
Based on that assumption, I added this:
serviceMonitor: create: true
To my scylla.values.yaml, hoping that that would do it, but when I try deploying locally with kind I get this error:
Upgrading release=scylla, chart=scylla/scylla Release "scylla" does not exist. Installing it now.
UPDATED RELEASES:
NAME CHART VERSION DURATION
cert-manager jetstack/cert-manager v1.13.3 24s
scylla-operator scylla/scylla-operator v1.11.1 24s
scylla-manager scylla/scylla-manager v1.11.1 2m4s
FAILED RELEASES:
NAME CHART VERSION DURATION
scylla scylla/scylla 1s
in ./helmfile.yaml: failed processing release scylla: command "/opt/homebrew/bin/helm" exited with non-zero status:
PATH:
/opt/homebrew/bin/helm
ARGS:
0: helm (4 bytes)
1: --kube-context (14 bytes)
2: kind-17381 (10 bytes)
3: upgrade (7 bytes)
4: --install (9 bytes)
5: scylla (6 bytes)
6: scylla/scylla (13 bytes)
7: --wait (6 bytes)
8: --timeout (9 bytes)
9: 900s (4 bytes)
10: --create-namespace (18 bytes)
11: --kube-context (14 bytes)
12: kind-17381 (10 bytes)
13: --namespace (11 bytes)
14: scylla (6 bytes)
15: --values (8 bytes)
16: /var/folders/gc/_n7v6nt9291_c_71zn24526c0000gn/T/helmfile4176172766/scylla-scylla-values-c74fc5797 (98 bytes)
17: --reset-values (14 bytes)
18: --history-max (13 bytes)
19: 10 (2 bytes)
ERROR:
exit status 1
EXIT STATUS
1
STDERR:
Error: unable to build kubernetes objects from release manifest: resource mapping not found for name: "scylla-service-monitor" namespace: "scylla" from "": no matches for kind "ServiceMonitor" in version "monitoring.coreos.com/v1" ensure CRDs are installed first
Which makes me think that I need to create the ServiceMonitor, or does that just mean that it’s failing to create the ServiceMonitor for some reason?
TL;DR my question is, given the current state of my deployment, how do I make my existing Prometheus deployment scrape the ScyllaDB metrics? And also, how can I get access to the dashboards that scylla-monitoring provides based on said metrics?
I also tried posting on the ScyllaDB slack but got no answer.
Thank you so much for any help!