How to Optimize ScyllaDB Performance for High Throughput Applications?

ScyllaDB
, , ,
1

Hey guys… :wave:

I am currently working on a high throughput application that requires low-latency and high concurrency for handling a large number of requests. I have chosen ScyllaDB due to its performance promises but I am now looking for advice on optimizing it further for my use case.

My application is built on a microservices architecture, where each service performs frequent reads and writes. I am already running a ScyllaDB cluster with 3 nodes, each equipped with NVMe SSDs and 64GB of RAM, but I feel there’s more I can do to fully exploit ScyllaDB’s potential.

Here are a few specific points I am looking for guidance on:

  1. Configuration tuning: What are the key settings (e.g., compaction, memtable_flush_writers, cache size, etc.) I should focus on to get the most out of ScyllaDB for high-throughput workloads?
  2. Best practices: Are there any community-recommended best practices for schema design or query optimization in ScyllaDB to minimize latency and maximize performance?
  3. Monitoring and metrics: What should I be tracking via ScyllaDB’s monitoring tools to identify and eliminate performance bottlenecks?

I also check this: https://forum.scylladb.com/t/scylladb-summit-day-2-continuing-the-high-performance-nosql-conversatlooker But I have not found any solution. Could anyone guide me about this? I have read through the documentation and a few case studies but would love to hear from the community about real-world experiences.

Thanks in advance!

Respected community member! :blush:

2

In general, there is no one-size-fits-all answer to these questions, as it depends mostly on your use case requirements. That said, I will be as generic as possible and hope it is somehow useful to you:

  • Configuration tuning: What are the key settings (e.g., compaction, memtable_flush_writers, cache size, etc.) I should focus on to get the most out of ScyllaDB for high-throughput workloads?

Ensure you walk through our Production Readiness Guidelines | ScyllaDB Docs . If your services are heavy-writers or heavy-readers, then there’s a nice table in https://docs.scylladb.com/stable/architecture/compaction/compaction-strategies.html which should be your guidance.

  • Best practices: Are there any community-recommended best practices for schema design or query optimization in ScyllaDB to minimize latency and maximize performance?

Yes, we recently hosted a Data Modeling for NoSQL Databases Masterclass which should be a good starting point. ScyllaDB University courses are also a great way to get started.

  • Monitoring and metrics: What should I be tracking via ScyllaDB’s monitoring tools to identify and eliminate performance bottlenecks?

Disk utilization, node health are some common KPIs. Get familiar with the Detailed dashboard though, there you can find granular P99 latencies, compaction information, foreground/background queue sizes, cache hit-rate among many other goodies.

All that said, our Database Performance at Scale: A Practical Guide covers most of these, and more in a consolidated way if you’d like to read on something. We organized it with recommendations a per-workload basis, driver settings, benchmarking, Monitoring KPIs, among many other topics which are probably somehow related to your question.

More recently, ScyllaDB In Action | By Bo Ingram, Staff Engineer at Discord got published, which is very ScyllaDB specific on its own.

Lastly, we do offer 1:1 Technical Consultations, so if you’d like to talk to someone, simply book a 30 min slot in ScyllaDB | Technical Strategy Session

Hope you find these useful and that you enjoy ScyllaDB!