In a Rust codebase I work with, we use
scylladb::Session::query() to retrieve some fixed-size results. However, sometimes we get the error:
Tombstones processed by unpaged query exceeds limit of 10000
(configured via query_tombstone_page_limit)
I’ve investigated this error and it seems that:
query_tombstone_page_limitis a parameter that tells ScyllaDB how many tombstones to process before cutting a new page of results in paged queries — presumably to limit the computational complexity of evaluating each page;
- When the query is unpaged, processing more than
query_tombstone_page_limittombstones in the course of the query causes the query to error out.
As a caller of the API, I can’t know statically how many tombstones my query will traverse. Does this mean that the only way to avoid triggering this error is to always use paged queries? If that’s true, I don’t see what the use case for unpaged queries is — doesn’t this imply that the user must always use paged queries, regardless of the size of their results?