Originally from the User Slack
@Shibu_ina:
[October 29th, 2024 9:36 PM] career.phuongnguyen: Hi everyone, i have question about lwt optimization done in scylla. Is the cql protocol extension just for better routing in client driver (disable host shuffling, retrying lwt on the same connection) or there is some optimization done on scylla cluster itself once it realizes that client driver knows about SCYLLA_LWT_OPTIMIZATION_META_BIT_MASK, i’m not familiar with C++ so i can not find the answer in scylla source code https://github.com/scylladb/scylladb/commit/6028588148c556151d764f2a987c276cb4a21eb0
@Felipe_Cardeneti_Mendes: For the LWT extension in question, https://groups.google.com/g/scylladb-dev/c/IQsbOpSDcTI/m/43kt-G0fAgAJ answers it:
The feature is meant to be further utilized by client drivers
to use primary replicas consistently when dealing with conditional
statements.
That said, other protocol extensions do exist. For tablet routing (so the client knows understands the new message propagated by the coordinator), and the per-partition-rate limiting error, alowing the driver to know when it needs to backoff and for how long if it hits it. https://github.com/scylladb/scylladb/blob/49d3e281d6a3ebbc23aca8def18408ace06ff9a1/transport/cql_protocol_extension.hh#L36
We have means to know whether a extension was set. For example, in https://github.com/scylladb/scylladb/commit/efc3953c0aafa2a49f610c11273725714a6b4f77#diff-5ddc61dd27612d0e2d48deba0[…]4d0b05de0f3b4126f4632236fR1284 we use:
if (!client_state.is_protocol_extension_set(cql_protocol_extension::RATE_LIMIT_ERROR)) {
return make_error(stream, exceptions::exception_code::CONFIG_ERROR, std::move(msg), tr_state);
}
and from https://github.com/scylladb/scylladb/blob/master/docs/dev/protocol-extensions.md the flow is described as:
• Client sends the OPTIONS request to the Scylla instance to get a list of protocol extensions that the server understands.
• Server sends the SUPPORTED message in reply to the OPTIONS request. The message body is a string multimap, in which keys describe different extensions and possibly one or more additional values specific to a particular extension (specified as distinct values under a feature key in the following form: ARG_NAME=VALUE).
• The client determines the set of compatible extensions which it is going to use in the current connection by intersecting known capabilities list with what it has received in SUPPORTED response.
• Client driver sends the STARTUP request with additional payload consisting of key-value pairs, each describing a negotiated extension.
• Server determines the set of compatible extensions by intersecting known list of protocol extensions with what it has received in STARTUP request.
So we have means to know it, but for LWT specifically the optimal routing is responsibility of the client driver.
@Shibu_ina: Thank you for the detailed answer! @\Felipe Cardeneti Mendes