How do I use `query()` safely?

ScyllaDB
, ,
1

In a Rust codebase I work with, we use scylladb::Session::query() to retrieve some fixed-size results. However, sometimes we get the error:

Tombstones processed by unpaged query exceeds limit of 10000
(configured via query_tombstone_page_limit)

I’ve investigated this error and it seems that:

  • query_tombstone_page_limit is a parameter that tells ScyllaDB how many tombstones to process before cutting a new page of results in paged queries — presumably to limit the computational complexity of evaluating each page;
  • When the query is unpaged, processing more than query_tombstone_page_limit tombstones in the course of the query causes the query to error out.

As a caller of the API, I can’t know statically how many tombstones my query will traverse. Does this mean that the only way to avoid triggering this error is to always use paged queries? If that’s true, I don’t see what the use case for unpaged queries is — doesn’t this imply that the user must always use paged queries, regardless of the size of their results?

2

Hello, you can check this article on how to reduce number of tombstones: How to flush old tombstones from a table | ScyllaDB Docs.

You can statistically assess how many tombstones there are based on your write load and compaction rate.