# Long connection times, Connection timeout using Rust driver, connection ports with Docker
**URL:** https://forum.scylladb.com/t/long-connection-times-connection-timeout-using-rust-driver-connection-ports-with-docker/4809
**Category:** ScyllaDB
**Tags:** drivers, troubleshooting, rust-driver, docker
**Created:** [May 6, 2025, 8:04am UTC](https://forum.scylladb.com/t/long-connection-times-connection-timeout-using-rust-driver-connection-ports-with-docker/4809 "2025-05-06T08:04:51Z")
**Posts on this page:** 2
**Page:** 1
### Author:  [@Guy](https://forum.scylladb.com/u/Guy)
#### Post date: [May 6, 2025, 8:04am UTC](https://forum.scylladb.com/t/long-connection-times-connection-timeout-using-rust-driver-connection-ports-with-docker/4809/1 "2025-05-06T08:04:51Z")
> [@](#):
>
> [**Originally from the User Slack**](https://slack.scylladb.com/)
>
>  **@Mattia:** Hello everyone
> I am trying ScyllaDB using the official rust driver. I have spawned 3 nodes in a cluster using docker compose and I build the `Session` from rust code.
> The problem I’m having is that the function that builds the connection blocks the executing until the timeout specified with `.connection_timeout(Duration::from_secs(..))` runs out, even if the connection is fine. For example if I set the timeout to 30 seconds I have to wait exactly 30 seconds before having a successful connection to the db.
>
> Is this the expected behavior? Thanks
>
>  **@Karol\_Baryła:** Definitely not an expected behavior. Could you share what is your OS, how exactly are you setting up the cluster, and how are you connecting to it?
>
>  **@Mattia:** Sure, thanks for the reply.
>
> System:
>
> ```auto
> Linux 6.14.3-zen1-1-zen x86-64 (Arch Linux)
>
> ```
>
> docker compose:
>
> ```auto
> services:
> scylla1:
> image: scylladb/scylla
> container_name: scylla1
> command: --smp 1
> ports:
> - "9042:9042"
>
> scylla2:
> image: scylladb/scylla
> container_name: scylla2
> command: --smp 1 --seeds=scylla1
>
> scylla3:
> image: scylladb/scylla
> container_name: scylla3
> command: --smp 1 --seeds=scylla1
>
> ```
>
> I’m using `docker-compose up -d` with `podman` as the backend.
>
> In Rust:
>
> ```auto
> async fn db_connect(conf: DBConf) -> Result {
> let known_nodes = conf.nodes();
>
> let session: Session = SessionBuilder::new()
> .known_nodes(known_nodes)
> .connection_timeout(Duration::from_secs(30))
> .compression(Some(Compression::Lz4))
> .build()
> .await?;
>
> Ok(session)
> }
>
> ```
>
> With `tracing` I see that it waits for x seconds before exiting the function, with x determined by the `connection_timeout` func
> Also, it is a debug build
>
>  **@Karol\_Baryła:** The problem is most likely that scylla2 and scylla3 are listening on localhost (in their own network namespaces), and broadcasting this address, so the driver can’t connect to them. For such docker setup you need to set up rpc address and listen address (and remove the “ports” section from scylla1). See for example the docker compose from tests of Rust Driver: [https://github.com/scylladb/scylla-rust-driver/blob/main/test/cluster/docker-compose.yml](https://github.com/scylladb/scylla-rust-driver/blob/main/test/cluster/docker-compose.yml)
>
> > GitHub: scylla-rust-driver/test/cluster/docker-compose.yml at main · scylladb/scylla-rust-driver
>
>  **@Mattia:** Thank you I will look into it. I got the docker compose from the docker’s hub page but what you are saying makes sense
> Just as a note: I’m only supplying “127.0.0.1:9042” as an address to the `SessionBuilder` so I thought it was sufficient to expose the port of one of the containers and everything would work.
>
>  **@Karol\_Baryła:** It would indeed allow the driver to connect to this one node. However the driver tries to connect to each node (each shard in fact). Connection process is roughly as follows:
> • One of provided contact points is chosen as control connection
> • Driver tries to connect to it
> • If successfull, driver then fetches information about cluster from system tables using this connection.
> • This allows the driver to learn about other nodes, to which it then tries to connect (this is the step that is hanging in your case)
> • After connecting, the Session is returned.
> If you want the driver to really connect only to this single node (which is usually a very bad idea, and only makes sense for tools like cqlsh) you could use [https://docs.rs/scylla/latest/scylla/policies/host\_filter/trait.HostFilter.html](https://docs.rs/scylla/latest/scylla/policies/host_filter/trait.HostFilter.html) ( [https://docs.rs/scylla/latest/scylla/client/session\_builder/type.SessionBuilder.html#method.host\_filter](https://docs.rs/scylla/latest/scylla/client/session_builder/type.SessionBuilder.html#method.host_filter) ) to prevent the driver from connecting to other nodes.
>
> > HostFilter in scylla::policies::host\_filter - Rust
>
> > SessionBuilder in scylla::client::session\_builder - Rust
>
>  **@Mattia:** Thank you so much for the explanation and your time. I will later try with the compose you have linked and see how it goes.
> For some reasons I get that the scylla1 node is unhealthy when trying to bring up the cluster. Any idea?
>
>  **@Karol\_Baryła:** Did you remove the ports section from scylla1?
>
>  **@Mattia:** Yes, I have completely removed the first file
>
>  **@Karol\_Baryła:** Can you show me the full docker compose?
>
>  **@Mattia:** Sure:
>
> ```auto
> networks
> public:
> name: scylla_node
> driver: bridge
> ipam:
> driver: default
> config:
> - subnet: 172.42.0.0/16
> services:
> node-scylla1:
> image: scylladb/scylla
> networks:
> public:
> ipv4_address: 172.42.0.2
> container_name: node-scylla1
> command: |
> --rpc-address 172.42.0.2
> --listen-address 172.42.0.2
> --seeds 172.42.0.2
> --skip-wait-for-gossip-to-settle 0
> --ring-delay-ms 0
> --smp 2
> --memory 1G
> healthcheck:
> test: ["CMD", "cqlsh", "node-scylla1", "-e", "select * from system.local WHERE key='local'"]
> interval: 5s
> timeout: 5s
> retries: 60
> node-scylla2:
> image: scylladb/scylla
> networks:
> public:
> ipv4_address: 172.42.0.3
> container_name: node-scylla2
> command: |
> --rpc-address 172.42.0.3
> --listen-address 172.42.0.3
> --seeds 172.42.0.2
> --skip-wait-for-gossip-to-settle 0
> --ring-delay-ms 0
> --smp 2
> --memory 1G
> healthcheck:
> test: ["CMD", "cqlsh", "node-scylla2", "-e", "select * from system.local WHERE key='local'"]
> interval: 5s
> timeout: 5s
> retries: 60
> depends_on:
> node-scylla1:
> condition: service_healthy
> node-scylla3:
> image: scylladb/scylla
> networks:
> public:
> ipv4_address: 172.42.0.4
> container_name: node-scylla3
> command: |
> --rpc-address 172.42.0.4
> --listen-address 172.42.0.4
> --seeds 172.42.0.2,172.42.0.3
> --skip-wait-for-gossip-to-settle 0
> --ring-delay-ms 0
> --smp 2
> --memory 1G
> healthcheck:
> test: ["CMD", "cqlsh", "node-scylla3", "-e", "select * from system.local WHERE key='local'"]
> interval: 5s
> timeout: 5s
> retries: 60
> depends_on:
> node-scylla2:
> condition: service_healthy
>
> ```
---
### Author:  [@Guy](https://forum.scylladb.com/u/Guy)
#### Post date: [May 7, 2025, 3:25am UTC](https://forum.scylladb.com/t/long-connection-times-connection-timeout-using-rust-driver-connection-ports-with-docker/4809/2 "2025-05-07T03:25:16Z")
> [@](#):
>
>  **@Mattia:** Hello, I am trying to make a contribution to ScyllaDB rust driver but I am unable to start the cluster to test the changes I have made.
> I am using the Makefile and running `make test` or `make ci` but both docker and podman hangs indefinitely (I think on the healthcheck of the first container).
>
> Can anyone help me with this? ([this is the docker compose used](https://github.com/scylladb/scylla-rust-driver/blob/00856129603b39676c9a31c26626d0ad5ec19c68/test/cluster/docker-compose.yml))
>
> > GitHub: scylla-rust-driver/test/cluster/docker-compose.yml at 00856129603b39676c9a31c26626d0ad5ec19c68 · scylladb/scylla-rust-driver
>
>  **@Karol\_Baryła:** Sorry for leaving you without response on a previous thread! Tbh this cluster sometimes (but rarely) also fails to start for me. In that case it is usually enough to try again (`make down && make up` ). I never really investigated why that happens. You can check logs of this node with `docker logs cluster-scylla1-1` and link them here (use pastebin / gist, or send a file), maybe we’ll find something there.
>
>  **@Mattia:** Unfortunately I have tried restarting it many times, I will try to share the logs now
> Here are the file for the logs from the container:
> [stdout.txt](https://forum.scylladb.com/uploads/short-url/s1HvULS4kaU36LmKc1JV2y1iLM7.txt) (2.9 KB)
> [stderr.txt](https://forum.scylladb.com/uploads/short-url/xlWF1a7lvQSW5Bc3C9xwJu9ZF88.txt) (109.1 KB)
>
> Even tough after 355 seconds the job stops due to unhealthy condition in scylla1 if I run `nodetools` I get the following status:
>
> ```auto
> docker exec -it cluster-scylla1-1 nodetool status
> Datacenter: datacenter1
> =======================
> Status=Up/Down
> |/ State=Normal/Leaving/Joining/Moving
> -- Address Load Tokens Owns Host ID Rack
> UN 172.42.0.2 269.25 KB 256 ? 5efe6b87-70b0-46a8-a92f-978e7067d027 rack1
>
> ```
>
>  **@Karol\_Baryła:** I have no idea why it doesn’t start. Scylla logs look fine I think. What happens if you execute the healthcheck command inside the container manually?
>
>  **@Mattia:** Doing
>
> 1. `docker exec -it cluster-scylla-1 /bin/bash`
> 2. `cqlsh -e "select * from system.local WHERE key='local'"`
> Returns a row correctly
> Do you think I should run this with root privileges?
> Like, `make test` and others
>
>  **@Karol\_Baryła:** Definitely without root. `docker inspect --format "{{json .State.Health }}" cluster-scylla-1 | jq` should show the output of healthcheck commands executed by docker - maybe it will give us some clue.
>
>  **@Mattia:** Thank you, this is rather telling:
>
> ```auto
> {"Status":"starting","FailingStreak":4,"Log":[{"Start":"2025-05-04T18:53:58.152812491+02:00","End":"2025-05-04T18:53:59.241185859+02:00","ExitCode":1,"Output":"Usage: cqlsh [options] [host [port]]\n\ncqlsh: error: '*' is not a valid port number.\n"},{"Start":"2025-05-04T18:54:04.877333293+02:00","End":"2025-05-04T18:54:05.386606825+02:00","ExitCode":1,"Output":"Usage: cqlsh [options] [host [port]]\n\ncqlsh: error: '*' is not a valid port number.\n"},{"Start":"2025-05-04T18:54:10.852202477+02:00","End":"2025-05-04T18:54:11.263149637+02:00","ExitCode":1,"Output":"Usage: cqlsh [options] [host [port]]\n\ncqlsh: error: '*' is not a valid port number.\n"},{"Start":"2025-05-04T18:54:16.874847316+02:00","End":"2025-05-04T18:54:17.332078002+02:00","ExitCode":1,"Output":"Usage: cqlsh [options] [host [port]]\n\ncqlsh: error: '*' is not a valid port number.\n"}]}
>
> ```
>
>  **@Karol\_Baryła:** I don’t see how it could possibly interpret `*` as port number in our compose file 😯 Did you modify the compose file? I see that your container name is different than mine, that may have something to do with it.
>
>  **@Mattia:**
>
> ```auto
> version: "3.7"
>
> networks:
> public:
> name: scylla_rust_driver_public
> driver: bridge
> ipam:
> driver: default
> config:
> - subnet: 172.42.0.0/16
> services:
> scylla1:
> image: scylladb/scylla
> networks:
> public:
> ipv4_address: 172.42.0.2
> command: |
> --rpc-address 172.42.0.2
> --listen-address 172.42.0.2
> --seeds 172.42.0.2
> --skip-wait-for-gossip-to-settle 0
> --ring-delay-ms 0
> --smp 2
> --memory 1G
> healthcheck:
> test: ["CMD", "cqlsh", "scylla1", "-e", "select * from system.local WHERE key='local'"]
> interval: 5s
> timeout: 5s
> retries: 60
> scylla2:
> image: scylladb/scylla
> networks:
> public:
> ipv4_address: 172.42.0.3
> command: |
> --rpc-address 172.42.0.3
> --listen-address 172.42.0.3
> --seeds 172.42.0.2
> --skip-wait-for-gossip-to-settle 0
> --ring-delay-ms 0
> --smp 2
> --memory 1G
> healthcheck:
> test: ["CMD", "cqlsh", "scylla2", "-e", "select * from system.local WHERE key='local'"]
> interval: 5s
> timeout: 5s
> retries: 60
> depends_on:
> scylla1:
> condition: service_healthy
> scylla3:
> image: scylladb/scylla
> networks:
> public:
> ipv4_address: 172.42.0.4
> command: |
> --rpc-address 172.42.0.4
> --listen-address 172.42.0.4
> --seeds 172.42.0.2,172.42.0.3
> --skip-wait-for-gossip-to-settle 0
> --ring-delay-ms 0
> --smp 2
> --memory 1G
> healthcheck:
> test: ["CMD", "cqlsh", "scylla3", "-e", "select * from system.local WHERE key='local'"]
> interval: 5s
> timeout: 5s
> retries: 60
> depends_on:
> scylla2:
> condition: service_healthy
>
> ```
>
> This is the one I am using, straight from the github repo
>
>  **@Karol\_Baryła:** What is the output of `docker --version`?
>
>  **@Mattia:** Docker version 28.1.1, build 4eba377327
>
> I also tried with podman with the same problem (podman version 5.4.2)
>
>  **@Karol\_Baryła:** In that case I’m unfortunately out of ideas, sorry 😞
>
>  **@Mattia:** Thank you for the help anyway! I think I will try to remove the docker-compose/podman layer. That might create some problems I guess
> Seems like I was still using podman, for some reasons it took precedence over docker when running the commands. With docker it works. (at least the first container is healthy, now the second seems to be hanging)
>  **@Mattia:** Thank you for the help anyway! I think I will try to remove the docker-compose/podman layer. That might create some problems I guess
>  **@Mattia:** Seems like I was still using podman, for some reasons it took precedence over docker when running the commands. With docker it works. (at least the first container is healthy, now the second seems to be hanging)