Originally from the User Slack
@Skunnyk: Hi I’m testing scylladb 5.4.6 which should solve lots of our LSA/memory problems due to excessive bloom filter size (thank you @Botond_Dénes for the help ), and I see that https://github.com/scylladb/scylladb/commit/1aedc7372de857bb80ef54aa0b87b21a217d82a3 is also present in the release.
Non-LSA Ram usage on cluster upgraded to 5.4.6 (from 5.4.x) is lower, and I try to understand this new “feature”. Does this means that “big” bloom filter are now reclaimed if they exceed 10% of the shard memory ? "Ratio of available memory for all in-memory components of SSTables in a shard beyond which the memory will be reclaimed from components until it falls back under the threshold. Currently, this limit is only enforced for bloom filters."
. Does this means that we can expect a bit more disk i/o because theses blooms filters are dropped ?
GitHub: Merge ‘[Backport 5.4] : Track and limit memory used by bloom filters’… · scylladb/scylladb@1aedc73
@avi: Yes, they get thrown out, you’ll get more disk I/O
you can adapt by increasing the ratio (will eat into your cache, so more I/O from that), or increasing the false positive chance (more false positives, so more I/O), or installing more RAM (less I/O)
@Skunnyk: Yes, that was my understanding Thank you !
Also, we upgraded to 5.4.6, and our bad_alloc and non-lsa problems are now gone