Scylla Manager 3.2.8 CQL SSL Timeout

ScyllaDB
,
1

Hello,

We are currently using ScyllaDB with Scylla Manager Open Source, and I’ve encountered an issue. When running the sctool status command, the CQL returns a TIMEOUT SSL error.

I couldn’t find specific information in the documentation regarding the correct configuration for SSL keys. While our application is configured to use SSL correctly with the necessary certificates, my attempts to apply the same configuration to the manager have been unsuccessful.

Could you please provide guidance on how to properly configure SSL for Scylla Manager?

Thank you.

Bundle:

scylla-manager --version
3.2.8-0.20240517.5f324acd2

scylla --version
5.4.6-0.20240418.10f137e367e3

3 Machines (for the Scylla Cluster) + 1 machine (for the Scylla Manager + scylla)
Everything runs on baremetal.

The Output:

ubuntu@scylla-manager:~$ sctool status -c REDACTED
Datacenter: REDACTED
╭────┬─────────────────────┬──────────┬────────────┬───────────┬──────┬──────────┬────────┬───────┬──────────────────────────────────────╮
│    │ CQL                 │ REST     │ Address    │ Uptime    │ CPUs │ Memory   │ Scylla │ Agent │ Host ID                              │
├────┼─────────────────────┼──────────┼────────────┼───────────┼──────┼──────────┼────────┼───────┼──────────────────────────────────────┤
│ UN │ TIMEOUT SSL (853ms) │ UP (0ms) │ 10.1.0.101 │ 25h27m45s │ 32   │ 125.736G │ 5.4.6  │ 3.2.8 │ REDACTED                             │
│ UN │ TIMEOUT SSL (852ms) │ UP (0ms) │ 10.1.0.102 │ 25h49m29s │ 32   │ 125.736G │ 5.4.6  │ 3.2.8 │ REDACTED                             │
│ UN │ TIMEOUT SSL (853ms) │ UP (1ms) │ 10.1.0.103 │ 25h39m15s │ 32   │ 125.736G │ 5.4.6  │ 3.2.8 │ REDACTED                             │
╰────┴─────────────────────┴──────────┴────────────┴───────────┴──────┴──────────┴────────┴───────┴──────────────────────────────────────╯

My last(current) attempt for the /etc/scylla-manager/scylla-manager.yaml on the manager machine.

Note: The scylla at 127.0.0.1 is a standalone node reserved for the manager.

http: 127.0.0.1:5080
https: 127.0.0.1:5443
database:
  hosts:
    - 127.0.0.1
  user: REDACTED
  password: REDACTED
  keyspace: scylla_manager
  replication_factor: 1
ssl:
  cert_file: /etc/scylla/ssl/certs/node_cadb.pem
  validate: false
2

Here is how I fixed my issue:

sctool cluster update --cluster REDACTED --force-non-ssl-session-port

The SSL configuration in /etc/scylla-manager/scylla-manager.yaml

ssl:
# CA certificate used to validate server cert. If not set will use he host's root CA set.
  cert_file: /etc/scylla-manager/ssl/certs/client_cadb.pem
#
# Verify the hostname and server cert.
  validate: false

# Client certificate and key in PEM format. It has to be provided when
# client_encryption_options.require_client_auth=true is set on server.
  user_cert_file: /etc/scylla-manager/ssl/certs/client.crt
  user_key_file: /etc/scylla-manager/ssl/certs/client.key

The reason I have to do this is that the cluster uses the port 9042 for SSL/TLS

References in scylladb documentation (I cannot post links…):

  • ctool/cluster.html#id15 (the sctool options)
  • configuration-parameters.html#confval-native_transport_port_ssl (description of why it uses 9042 instead of 9142)
1 Like