Commit Log Schema Separation

ScyllaDB
1

We recently noticed commitlog changes for new nodes, after upgrading to 5.1, and I’m not sure what to make of it.

There is a new configuration force_schema_commit_log that says it will enable a separate commitlog immediately when enabled, as compared to enabling it on the first reboot. This is set to true by default.

I however, cannot find any documentation for what this commitlog separation on first boot is or why we need it, or even if I can enable/disable it, outside of the first boot. In the 5.1.0 release notes the main references to schema changes are Raft, which is still experimental, so I assume this isn’t Raft related, or it wouldn’t default to true.

Should we have done a full cluster reboot after the 5.1 upgrade to enable this feature, and then this is all a moot point? How can I verify this is enabled on a node, besides commitlog errors on boot?

2

You’ll have files of the form /var/lib/scylla/commilog/SchemaLog*.log. No action is required on your part.

The idea behind it is to give schema and topology a commitlog that is more durable (commitlog_sync = batch instead of periodic), and to give schema and topology higher priority over your writes, so you can’t flood the database with writes and watch it malfunction as its internal metadata operations are delayed indefinitely.

3

Ok, so we don’t even need to reboot after the cluster was upgraded to 5.1? It should already be active on all nodes due to the reboot while upgrading? I checked a few nodes that haven’t been restarted and didn’t see any of the mentioned files. Though ones that were restarted I do see. Should I just give everything a rolling reboot to make sure it’s enabled?

4

Perhaps it does need a rolling restart, though it’s okay to delay it.