Basic rate limiter using ScyllaDB. How?

ScyllaDB
1

I want to use Scylla to write simple rate limiter (because I already use Scylla and don’t want to add another DB just for that).

Counters look like a good tool for that, however they don’t have any kind of expiration. I already solved similar problem, where I need a limit per day. I basically create table every day and use it for writes, then create new one and use that (the old is deleted via cronjob).

Now I want to rate limit requests per minute, so it’s not a good solution. I can use something like that:

create table rate_limiter (
  id ascii,
  slot int,
  cu_used counter,
  PRIMARY KEY (id, slot)
);

Where slot is basically a time slot (say floor(unixtime/60)). It would be enough, but of course I need some expiration. I could probably do the same, create new table every day, delete the old one. However, this looks messy and I don’t like that at all.

  • I could probably use something other than counters, but only the have increments.
  • Inserting a row per request looks like too much overhead for rate limiting

Are there any better solutions?

2

The underlying problem here is that counters don’t support expiry. So you either have to switch to a data type which does support expiry (basically anything apart from counters), or you need another cron job which removes rows from this table from time-to-time.