Compaction Storm slows down Scylla

horschi · October 6, 2024, 8:45pm

I stand corrected: I just saw a 3MB collection warning in the logs.

horschi · October 6, 2024, 8:45pm

We’ll try to decode tomorrow.

Womsel · October 7, 2024, 1:13pm

We haven’t observed the behaviour since the last restart of scylla, then we added a few more nodes, got them bootstrapped and they started to serve.

Switched the logger for commitlog from “info” to “trace” on all nodes, and have seen something interesting. Specifically those added nodes were busy doing some weird commitlog-activity, that kept going on-and-on, here an excerpt.

After another restart of those new nodes only, it’s silent.

Oct 07 12:02:47 o-p-L3-8 scylla[32999]:  [shard 2:strm] commitlog - Segment HintsLog-3-36028797047381970.log is no longer active and will submitted for delete now
Oct 07 12:02:47 o-p-L3-8 scylla[32999]:  [shard 2:strm] commitlog - Discarding segments [/var/lib/scylla/hints/2/d01ea9d0-787f-4aa5-9235-6f9b3d424063/HintsLog-3-36028797047381970.log (33554432) (Force Delete)]
Oct 07 12:02:47 o-p-L3-8 scylla[32999]:  [shard 2:strm] commitlog - Deleting segment file /var/lib/scylla/hints/2/d01ea9d0-787f-4aa5-9235-6f9b3d424063/HintsLog-3-36028797047381970.log
Oct 07 12:02:47 o-p-L3-8 scylla[32999]:  [shard 2:strm] commitlog - Commitlog shutdown complete
Oct 07 12:02:47 o-p-L3-8 scylla[32999]:  [shard 2:strm] commitlog - Commitlog /var/lib/scylla/hints/2/d01ea9d0-787f-4aa5-9235-6f9b3d424063 disposed
Oct 07 12:02:47 o-p-L3-8 scylla[32999]:  [shard 2:strm] commitlog - Commitlog /var/lib/scylla/hints/2/d01ea9d0-787f-4aa5-9235-6f9b3d424063 maximum disk size: 22 MB / cpu (6 cpus)
Oct 07 12:02:47 o-p-L3-8 scylla[32999]:  [shard 2:strm] commitlog - Delaying timer loop 3334 ms
Oct 07 12:02:47 o-p-L3-8 scylla[32999]:  [shard 2:main] commitlog - Created new segment HintsLog-3-36028797047381970.log
Oct 07 12:02:47 o-p-L3-8 scylla[32999]:  [shard 0: gms] commitlog - Allocated 128 k buffer
Oct 07 12:02:47 o-p-L3-8 scylla[32999]:  [shard 0: gms] commitlog - Writing 1 entries, 4096 k in 122220544 -> 122224640
Oct 07 12:02:47 o-p-L3-8 scylla[32999]:  [shard 0: gms] commitlog - Allocated 128 k buffer
Oct 07 12:02:47 o-p-L3-8 scylla[32999]:  [shard 0:sclg] commitlog - Final write of 4096 to SchemaLog-3-28418171.log: 4096/4096 bytes at 122220544
Oct 07 12:02:47 o-p-L3-8 scylla[32999]:  [shard 0: gms] commitlog - SchemaLog-3-28418171.log synced to 122224640
Oct 07 12:02:47 o-p-L3-8 scylla[32999]:  [shard 0: gms] commitlog - Writing 1 entries, 4096 k in 122224640 -> 122228736
Oct 07 12:02:47 o-p-L3-8 scylla[32999]:  [shard 0:sclg] commitlog - Final write of 4096 to SchemaLog-3-28418171.log: 4096/4096 bytes at 122224640
Oct 07 12:02:47 o-p-L3-8 scylla[32999]:  [shard 0: gms] commitlog - SchemaLog-3-28418171.log synced to 122228736
Oct 07 12:02:47 o-p-L3-8 scylla[32999]:  [shard 0: gms] commitlog - Allocated 128 k buffer
Oct 07 12:02:47 o-p-L3-8 scylla[32999]:  [shard 0: gms] commitlog - Writing 1 entries, 4096 k in 122228736 -> 122232832
Oct 07 12:02:47 o-p-L3-8 scylla[32999]:  [shard 0: gms] commitlog - Allocated 128 k buffer
Oct 07 12:02:47 o-p-L3-8 scylla[32999]:  [shard 0:sclg] commitlog - Final write of 4096 to SchemaLog-3-28418171.log: 4096/4096 bytes at 122228736
Oct 07 12:02:47 o-p-L3-8 scylla[32999]:  [shard 0: gms] commitlog - SchemaLog-3-28418171.log synced to 122232832
Oct 07 12:02:47 o-p-L3-8 scylla[32999]:  [shard 0: gms] commitlog - Writing 9 entries, 4096 k in 122232832 -> 122236928
Oct 07 12:02:47 o-p-L3-8 scylla[32999]:  [shard 0:sclg] commitlog - Final write of 4096 to SchemaLog-3-28418171.log: 4096/4096 bytes at 122232832
Oct 07 12:02:47 o-p-L3-8 scylla[32999]:  [shard 0: gms] commitlog - SchemaLog-3-28418171.log synced to 122236928
Oct 07 12:02:47 o-p-L3-8 scylla[32999]:  [shard 1:stmt] commitlog - Writing 302 entries, 131072 k in 5058560 -> 5189632
Oct 07 12:02:47 o-p-L3-8 scylla[32999]:  [shard 1:stmt] commitlog - Allocated 128 k buffer
Oct 07 12:02:47 o-p-L3-8 scylla[32999]:  [shard 1:clog] commitlog - Final write of 131072 to CommitLog-3-18014398537900605.log: 131072/131072 bytes at 5058560
Oct 07 12:02:47 o-p-L3-8 scylla[32999]:  [shard 3:stmt] commitlog - Writing 258 entries, 131072 k in 28348416 -> 28479488
Oct 07 12:02:47 o-p-L3-8 scylla[32999]:  [shard 3:stmt] commitlog - Allocated 128 k buffer
Oct 07 12:02:47 o-p-L3-8 scylla[32999]:  [shard 3:clog] commitlog - Final write of 131072 to CommitLog-3-54043195556864646.log: 131072/131072 bytes at 28348416
Oct 07 12:02:47 o-p-L3-8 scylla[32999]:  [shard 0: gms] commitlog - Allocated 128 k buffer
Oct 07 12:02:47 o-p-L3-8 scylla[32999]:  [shard 0: gms] commitlog - Writing 1 entries, 4096 k in 122236928 -> 122241024
Oct 07 12:02:47 o-p-L3-8 scylla[32999]:  [shard 0: gms] commitlog - Allocated 128 k buffer
Oct 07 12:02:47 o-p-L3-8 scylla[32999]:  [shard 0:sclg] commitlog - Final write of 4096 to SchemaLog-3-28418171.log: 4096/4096 bytes at 122236928
Oct 07 12:02:47 o-p-L3-8 scylla[32999]:  [shard 0: gms] commitlog - SchemaLog-3-28418171.log synced to 122241024
Oct 07 12:02:47 o-p-L3-8 scylla[32999]:  [shard 0: gms] commitlog - Writing 4 entries, 4096 k in 122241024 -> 122245120
Oct 07 12:02:47 o-p-L3-8 scylla[32999]:  [shard 0:sclg] commitlog - Final write of 4096 to SchemaLog-3-28418171.log: 4096/4096 bytes at 122241024
Oct 07 12:02:47 o-p-L3-8 scylla[32999]:  [shard 0: gms] commitlog - SchemaLog-3-28418171.log synced to 122245120
Oct 07 12:02:47 o-p-L3-8 scylla[32999]:  [shard 5:stmt] commitlog - Writing 360 entries, 131072 k in 34680832 -> 34811904
Oct 07 12:02:47 o-p-L3-8 scylla[32999]:  [shard 5:stmt] commitlog - Allocated 128 k buffer
Oct 07 12:02:47 o-p-L3-8 scylla[32999]:  [shard 5:clog] commitlog - Final write of 131072 to CommitLog-3-90071992575828770.log: 131072/131072 bytes at 34680832
Oct 07 12:02:47 o-p-L3-8 scylla[32999]:  [shard 0: gms] commitlog - Allocated 128 k buffer
Oct 07 12:02:47 o-p-L3-8 scylla[32999]:  [shard 0: gms] commitlog - Writing 1 entries, 4096 k in 122245120 -> 122249216
Oct 07 12:02:47 o-p-L3-8 scylla[32999]:  [shard 0: gms] commitlog - Allocated 128 k buffer
Oct 07 12:02:47 o-p-L3-8 scylla[32999]:  [shard 0:sclg] commitlog - Final write of 4096 to SchemaLog-3-28418171.log: 4096/4096 bytes at 122245120
Oct 07 12:02:47 o-p-L3-8 scylla[32999]:  [shard 0: gms] commitlog - SchemaLog-3-28418171.log synced to 122249216
Oct 07 12:02:47 o-p-L3-8 scylla[32999]:  [shard 0: gms] commitlog - Writing 4 entries, 4096 k in 122249216 -> 122253312
Oct 07 12:02:47 o-p-L3-8 scylla[32999]:  [shard 0:sclg] commitlog - Final write of 4096 to SchemaLog-3-28418171.log: 4096/4096 bytes at 122249216
Oct 07 12:02:47 o-p-L3-8 scylla[32999]:  [shard 0: gms] commitlog - SchemaLog-3-28418171.log synced to 122253312
Oct 07 12:02:47 o-p-L3-8 scylla[32999]:  [shard 0: gms] commitlog - Allocated 128 k buffer
Oct 07 12:02:47 o-p-L3-8 scylla[32999]:  [shard 0: gms] commitlog - Writing 1 entries, 4096 k in 122253312 -> 122257408
Oct 07 12:02:47 o-p-L3-8 scylla[32999]:  [shard 0: gms] commitlog - Allocated 128 k buffer
Oct 07 12:02:47 o-p-L3-8 scylla[32999]:  [shard 0:sclg] commitlog - Final write of 4096 to SchemaLog-3-28418171.log: 4096/4096 bytes at 122253312
Oct 07 12:02:47 o-p-L3-8 scylla[32999]:  [shard 0: gms] commitlog - SchemaLog-3-28418171.log synced to 122257408
Oct 07 12:02:47 o-p-L3-8 scylla[32999]:  [shard 0: gms] commitlog - Writing 10 entries, 4096 k in 122257408 -> 122261504
Oct 07 12:02:47 o-p-L3-8 scylla[32999]:  [shard 0:sclg] commitlog - Final write of 4096 to SchemaLog-3-28418171.log: 4096/4096 bytes at 122257408
Oct 07 12:02:47 o-p-L3-8 scylla[32999]:  [shard 0: gms] commitlog - SchemaLog-3-28418171.log synced to 122261504
Oct 07 12:02:47 o-p-L3-8 scylla[32999]:  [shard 0: gms] commitlog - Allocated 128 k buffer
Oct 07 12:02:47 o-p-L3-8 scylla[32999]:  [shard 0: gms] commitlog - Writing 1 entries, 4096 k in 122261504 -> 122265600
Oct 07 12:02:47 o-p-L3-8 scylla[32999]:  [shard 0: gms] commitlog - Allocated 128 k buffer
Oct 07 12:02:47 o-p-L3-8 scylla[32999]:  [shard 0:sclg] commitlog - Final write of 4096 to SchemaLog-3-28418171.log: 4096/4096 bytes at 122261504
Oct 07 12:02:47 o-p-L3-8 scylla[32999]:  [shard 0: gms] commitlog - SchemaLog-3-28418171.log synced to 122265600
Oct 07 12:02:47 o-p-L3-8 scylla[32999]:  [shard 0: gms] commitlog - Writing 2 entries, 4096 k in 122265600 -> 122269696
Oct 07 12:02:47 o-p-L3-8 scylla[32999]:  [shard 0:sclg] commitlog - Final write of 4096 to SchemaLog-3-28418171.log: 4096/4096 bytes at 122265600
Oct 07 12:02:47 o-p-L3-8 scylla[32999]:  [shard 0: gms] commitlog - SchemaLog-3-28418171.log synced to 122269696
Oct 07 12:02:48 o-p-L3-8 scylla[32999]:  [shard 4:stmt] commitlog - Writing 234 entries, 102400 k in 4366336 -> 4468736
Oct 07 12:02:48 o-p-L3-8 scylla[32999]:  [shard 4:stmt] commitlog - Allocated 128 k buffer
Oct 07 12:02:48 o-p-L3-8 scylla[32999]:  [shard 4:clog] commitlog - Final write of 102400 to CommitLog-3-72057594066346605.log: 102400/102400 bytes at 4366336
Oct 07 12:02:48 o-p-L3-8 scylla[32999]:  [shard 2:stmt] commitlog - Writing 307 entries, 131072 k in 32587776 -> 32718848
Oct 07 12:02:48 o-p-L3-8 scylla[32999]:  [shard 2:stmt] commitlog - Allocated 128 k buffer
Oct 07 12:02:48 o-p-L3-8 scylla[32999]:  [shard 2:clog] commitlog - Final write of 131072 to CommitLog-3-36028797047382674.log: 131072/131072 bytes at 32587776
Oct 07 12:02:48 o-p-L3-8 scylla[32999]:  [shard 0: gms] commitlog - Allocated 128 k buffer
Oct 07 12:02:48 o-p-L3-8 scylla[32999]:  [shard 0: gms] commitlog - Writing 1 entries, 4096 k in 122269696 -> 122273792
Oct 07 12:02:48 o-p-L3-8 scylla[32999]:  [shard 0: gms] commitlog - Allocated 128 k buffer
Oct 07 12:02:48 o-p-L3-8 scylla[32999]:  [shard 0:sclg] commitlog - Final write of 4096 to SchemaLog-3-28418171.log: 4096/4096 bytes at 122269696
Oct 07 12:02:48 o-p-L3-8 scylla[32999]:  [shard 0: gms] commitlog - SchemaLog-3-28418171.log synced to 122273792
Oct 07 12:02:48 o-p-L3-8 scylla[32999]:  [shard 0: gms] commitlog - Writing 25 entries, 8192 k in 122273792 -> 122281984
Oct 07 12:02:48 o-p-L3-8 scylla[32999]:  [shard 0:sclg] commitlog - Final write of 8192 to SchemaLog-3-28418171.log: 8192/8192 bytes at 122273792
Oct 07 12:02:48 o-p-L3-8 scylla[32999]:  [shard 0: gms] commitlog - SchemaLog-3-28418171.log synced to 122281984
Oct 07 12:02:48 o-p-L3-8 scylla[32999]:  [shard 0: gms] commitlog - Allocated 128 k buffer
Oct 07 12:02:48 o-p-L3-8 scylla[32999]:  [shard 0: gms] commitlog - Writing 1 entries, 4096 k in 122281984 -> 122286080
Oct 07 12:02:48 o-p-L3-8 scylla[32999]:  [shard 0: gms] commitlog - Allocated 128 k buffer
Oct 07 12:02:48 o-p-L3-8 scylla[32999]:  [shard 0:sclg] commitlog - Final write of 4096 to SchemaLog-3-28418171.log: 4096/4096 bytes at 122281984
Oct 07 12:02:48 o-p-L3-8 scylla[32999]:  [shard 0: gms] commitlog - SchemaLog-3-28418171.log synced to 122286080
Oct 07 12:02:48 o-p-L3-8 scylla[32999]:  [shard 0: gms] commitlog - Writing 2 entries, 4096 k in 122286080 -> 122290176
Oct 07 12:02:48 o-p-L3-8 scylla[32999]:  [shard 0:sclg] commitlog - Final write of 4096 to SchemaLog-3-28418171.log: 4096/4096 bytes at 122286080
Oct 07 12:02:48 o-p-L3-8 scylla[32999]:  [shard 0: gms] commitlog - SchemaLog-3-28418171.log synced to 122290176
Oct 07 12:02:48 o-p-L3-8 scylla[32999]:  [shard 0: gms] commitlog - Allocated 128 k buffer
Oct 07 12:02:48 o-p-L3-8 scylla[32999]:  [shard 0: gms] commitlog - Writing 1 entries, 4096 k in 122290176 -> 122294272
Oct 07 12:02:48 o-p-L3-8 scylla[32999]:  [shard 0:sclg] commitlog - Final write of 4096 to SchemaLog-3-28418171.log: 4096/4096 bytes at 122290176
Oct 07 12:02:48 o-p-L3-8 scylla[32999]:  [shard 0: gms] commitlog - SchemaLog-3-28418171.log synced to 122294272
Oct 07 12:02:48 o-p-L3-8 scylla[32999]:  [shard 1:stmt] commitlog - Writing 217 entries, 131072 k in 5189632 -> 5320704
Oct 07 12:02:48 o-p-L3-8 scylla[32999]:  [shard 1:stmt] commitlog - Allocated 128 k buffer
Oct 07 12:02:48 o-p-L3-8 scylla[32999]:  [shard 1:clog] commitlog - Final write of 131072 to CommitLog-3-18014398537900605.log: 131072/131072 bytes at 5189632
Oct 07 12:02:49 o-p-L3-8 scylla[32999]:  [shard 0: gms] commitlog - Allocated 128 k buffer
Oct 07 12:02:49 o-p-L3-8 scylla[32999]:  [shard 0: gms] commitlog - Writing 1 entries, 4096 k in 122294272 -> 122298368
Oct 07 12:02:49 o-p-L3-8 scylla[32999]:  [shard 0: gms] commitlog - Allocated 128 k buffer
Oct 07 12:02:49 o-p-L3-8 scylla[32999]:  [shard 0:sclg] commitlog - Final write of 4096 to SchemaLog-3-28418171.log: 4096/4096 bytes at 122294272
Oct 07 12:02:49 o-p-L3-8 scylla[32999]:  [shard 0: gms] commitlog - SchemaLog-3-28418171.log synced to 122298368
Oct 07 12:02:49 o-p-L3-8 scylla[32999]:  [shard 0: gms] commitlog - Writing 1 entries, 4096 k in 122298368 -> 122302464
Oct 07 12:02:49 o-p-L3-8 scylla[32999]:  [shard 0:sclg] commitlog - Final write of 4096 to SchemaLog-3-28418171.log: 4096/4096 bytes at 122298368
Oct 07 12:02:49 o-p-L3-8 scylla[32999]:  [shard 0: gms] commitlog - SchemaLog-3-28418171.log synced to 122302464
Oct 07 12:02:49 o-p-L3-8 scylla[32999]:  [shard 0: gms] commitlog - Allocated 128 k buffer
Oct 07 12:02:49 o-p-L3-8 scylla[32999]:  [shard 0: gms] commitlog - Writing 1 entries, 4096 k in 122302464 -> 122306560
Oct 07 12:02:49 o-p-L3-8 scylla[32999]:  [shard 0:sclg] commitlog - Final write of 4096 to SchemaLog-3-28418171.log: 4096/4096 bytes at 122302464
Oct 07 12:02:49 o-p-L3-8 scylla[32999]:  [shard 0: gms] commitlog - SchemaLog-3-28418171.log synced to 122306560
Oct 07 12:02:49 o-p-L3-8 scylla[32999]:  [shard 0:stmt] commitlog - Writing 376 entries, 131072 k in 63127552 -> 63258624
Oct 07 12:02:49 o-p-L3-8 scylla[32999]:  [shard 0:stmt] commitlog - Allocated 128 k buffer
Oct 07 12:02:49 o-p-L3-8 scylla[32999]:  [shard 0:clog] commitlog - Final write of 131072 to CommitLog-3-28418828.log: 131072/131072 bytes at 63127552
Oct 07 12:02:49 o-p-L3-8 scylla[32999]:  [shard 0: gms] commitlog - Allocated 128 k buffer
Oct 07 12:02:49 o-p-L3-8 scylla[32999]:  [shard 0: gms] commitlog - Writing 1 entries, 4096 k in 122306560 -> 122310656
Oct 07 12:02:49 o-p-L3-8 scylla[32999]:  [shard 0: gms] commitlog - Allocated 128 k buffer
Oct 07 12:02:49 o-p-L3-8 scylla[32999]:  [shard 0:sclg] commitlog - Final write of 4096 to SchemaLog-3-28418171.log: 4096/4096 bytes at 122306560
Oct 07 12:02:49 o-p-L3-8 scylla[32999]:  [shard 0: gms] commitlog - SchemaLog-3-28418171.log synced to 122310656
Oct 07 12:02:49 o-p-L3-8 scylla[32999]:  [shard 0: gms] commitlog - Writing 3 entries, 4096 k in 122310656 -> 122314752
Oct 07 12:02:49 o-p-L3-8 scylla[32999]:  [shard 0:sclg] commitlog - Final write of 4096 to SchemaLog-3-28418171.log: 4096/4096 bytes at 122310656
Oct 07 12:02:49 o-p-L3-8 scylla[32999]:  [shard 0: gms] commitlog - SchemaLog-3-28418171.log synced to 122314752                                                                                                                                    Oct 07 12:02:49 o-p-L3-8 scylla[32999]:  [shard 3:stmt] commitlog - Writing 252 entries, 131072 k in 28479488 -> 28610560
Oct 07 12:02:49 o-p-L3-8 scylla[32999]:  [shard 3:stmt] commitlog - Allocated 128 k buffer
Oct 07 12:02:49 o-p-L3-8 scylla[32999]:  [shard 3:clog] commitlog - Final write of 131072 to CommitLog-3-54043195556864646.log: 131072/131072 bytes at 28479488
Oct 07 12:02:49 o-p-L3-8 scylla[32999]:  [shard 5:stmt] commitlog - Writing 293 entries, 131072 k in 34811904 -> 34942976
Oct 07 12:02:49 o-p-L3-8 scylla[32999]:  [shard 5:stmt] commitlog - Allocated 128 k buffer
Oct 07 12:02:49 o-p-L3-8 scylla[32999]:  [shard 5:clog] commitlog - Final write of 131072 to CommitLog-3-90071992575828770.log: 131072/131072 bytes at 34811904
Oct 07 12:02:49 o-p-L3-8 scylla[32999]:  [shard 0: gms] commitlog - Allocated 128 k buffer
Oct 07 12:02:49 o-p-L3-8 scylla[32999]:  [shard 0: gms] commitlog - Writing 1 entries, 4096 k in 122314752 -> 122318848
Oct 07 12:02:49 o-p-L3-8 scylla[32999]:  [shard 0: gms] commitlog - Allocated 128 k buffer
Oct 07 12:02:49 o-p-L3-8 scylla[32999]:  [shard 0:sclg] commitlog - Final write of 4096 to SchemaLog-3-28418171.log: 4096/4096 bytes at 122314752
Oct 07 12:02:49 o-p-L3-8 scylla[32999]:  [shard 0: gms] commitlog - SchemaLog-3-28418171.log synced to 122318848
Oct 07 12:02:49 o-p-L3-8 scylla[32999]:  [shard 0: gms] commitlog - Writing 12 entries, 4096 k in 122318848 -> 122322944
Oct 07 12:02:49 o-p-L3-8 scylla[32999]:  [shard 0:sclg] commitlog - Final write of 4096 to SchemaLog-3-28418171.log: 4096/4096 bytes at 122318848
Oct 07 12:02:49 o-p-L3-8 scylla[32999]:  [shard 0: gms] commitlog - SchemaLog-3-28418171.log synced to 122322944
Oct 07 12:02:49 o-p-L3-8 scylla[32999]:  [shard 0: gms] commitlog - Allocated 128 k buffer
Oct 07 12:02:49 o-p-L3-8 scylla[32999]:  [shard 0: gms] commitlog - Writing 1 entries, 4096 k in 122322944 -> 122327040
Oct 07 12:02:49 o-p-L3-8 scylla[32999]:  [shard 0: gms] commitlog - Allocated 128 k buffer
Oct 07 12:02:49 o-p-L3-8 scylla[32999]:  [shard 0:sclg] commitlog - Final write of 4096 to SchemaLog-3-28418171.log: 4096/4096 bytes at 122322944
Oct 07 12:02:49 o-p-L3-8 scylla[32999]:  [shard 0: gms] commitlog - SchemaLog-3-28418171.log synced to 122327040
Oct 07 12:02:49 o-p-L3-8 scylla[32999]:  [shard 0: gms] commitlog - Writing 1 entries, 4096 k in 122327040 -> 122331136
Oct 07 12:02:49 o-p-L3-8 scylla[32999]:  [shard 0: gms] commitlog - Allocated 128 k buffer
Oct 07 12:02:49 o-p-L3-8 scylla[32999]:  [shard 0:sclg] commitlog - Final write of 4096 to SchemaLog-3-28418171.log: 4096/4096 bytes at 122327040
Oct 07 12:02:49 o-p-L3-8 scylla[32999]:  [shard 0: gms] commitlog - SchemaLog-3-28418171.log synced to 122331136
Oct 07 12:02:49 o-p-L3-8 scylla[32999]:  [shard 0: gms] commitlog - Writing 10 entries, 4096 k in 122331136 -> 122335232
Oct 07 12:02:49 o-p-L3-8 scylla[32999]:  [shard 0: gms] commitlog - Allocated 128 k buffer
Oct 07 12:02:49 o-p-L3-8 scylla[32999]:  [shard 0:sclg] commitlog - Final write of 4096 to SchemaLog-3-28418171.log: 4096/4096 bytes at 122331136
Oct 07 12:02:49 o-p-L3-8 scylla[32999]:  [shard 0: gms] commitlog - SchemaLog-3-28418171.log synced to 122335232
Oct 07 12:02:49 o-p-L3-8 scylla[32999]:  [shard 0: gms] commitlog - Writing 4 entries, 4096 k in 122335232 -> 122339328
Oct 07 12:02:49 o-p-L3-8 scylla[32999]:  [shard 0: gms] commitlog - Allocated 128 k buffer
Oct 07 12:02:49 o-p-L3-8 scylla[32999]:  [shard 0:sclg] commitlog - Final write of 4096 to SchemaLog-3-28418171.log: 4096/4096 bytes at 122335232
Oct 07 12:02:49 o-p-L3-8 scylla[32999]:  [shard 0: gms] commitlog - SchemaLog-3-28418171.log synced to 122339328
Oct 07 12:02:49 o-p-L3-8 scylla[32999]:  [shard 0: gms] commitlog - Writing 1 entries, 4096 k in 122339328 -> 122343424
Oct 07 12:02:49 o-p-L3-8 scylla[32999]:  [shard 0:sclg] commitlog - Final write of 4096 to SchemaLog-3-28418171.log: 4096/4096 bytes at 122339328
Oct 07 12:02:49 o-p-L3-8 scylla[32999]:  [shard 0: gms] commitlog - SchemaLog-3-28418171.log synced to 122343424
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 0:strm] commitlog - Issuing sync for all segments
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 0:strm] commitlog - Checking for unused segments (0 active)
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 0:strm] commitlog - Issuing shutdown for all segments
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 0:strm] commitlog - Checking for unused segments (0 active)
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 0:strm] commitlog - Segment HintsLog-3-28418000.log is no longer active and will submitted for delete now
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 0:strm] commitlog - Discarding segments [/var/lib/scylla/hints/0/17b6fc29-812b-4812-bc3a-ecfa256828f2/HintsLog-3-28418000.log (33554432) (Force Delete)]
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 0:strm] commitlog - Issuing sync for all segments
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 0:strm] commitlog - Checking for unused segments (0 active)
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 0:strm] commitlog - Issuing shutdown for all segments
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 0:strm] commitlog - Checking for unused segments (0 active)
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 0:strm] commitlog - Segment HintsLog-3-28417993.log is no longer active and will submitted for delete now
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 0:strm] commitlog - Discarding segments [/var/lib/scylla/hints/0/51f9ec26-776f-4f66-96f0-51018af4e9b7/HintsLog-3-28417993.log (33554432) (Force Delete)]
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 0:strm] commitlog - Issuing sync for all segments
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 0:strm] commitlog - Checking for unused segments (0 active)
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 0:strm] commitlog - Issuing shutdown for all segments
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 0:strm] commitlog - Checking for unused segments (0 active)
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 0:strm] commitlog - Segment HintsLog-3-28418003.log is no longer active and will submitted for delete now
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 0:strm] commitlog - Discarding segments [/var/lib/scylla/hints/0/81ec2a79-17b5-451e-b6d1-960aab842f3f/HintsLog-3-28418003.log (33554432) (Force Delete)]
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 0:strm] commitlog - Issuing sync for all segments
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 0:strm] commitlog - Checking for unused segments (0 active)
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 0:strm] commitlog - Issuing shutdown for all segments
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 0:strm] commitlog - Checking for unused segments (0 active)
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 0:strm] commitlog - Segment HintsLog-3-28417994.log is no longer active and will submitted for delete now
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 0:strm] commitlog - Discarding segments [/var/lib/scylla/hints/0/dcfb91cf-5d0a-436b-b71a-a554c1bd1bc5/HintsLog-3-28417994.log (33554432) (Force Delete)]
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 0:strm] commitlog - Deleting segment file /var/lib/scylla/hints/0/17b6fc29-812b-4812-bc3a-ecfa256828f2/HintsLog-3-28418000.log
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 0:strm] commitlog - Deleting segment file /var/lib/scylla/hints/0/51f9ec26-776f-4f66-96f0-51018af4e9b7/HintsLog-3-28417993.log
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 0:strm] commitlog - Deleting segment file /var/lib/scylla/hints/0/81ec2a79-17b5-451e-b6d1-960aab842f3f/HintsLog-3-28418003.log
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 0:strm] commitlog - Deleting segment file /var/lib/scylla/hints/0/dcfb91cf-5d0a-436b-b71a-a554c1bd1bc5/HintsLog-3-28417994.log
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 0:strm] commitlog - Commitlog shutdown complete
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 0:strm] commitlog - Commitlog /var/lib/scylla/hints/0/17b6fc29-812b-4812-bc3a-ecfa256828f2 disposed
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 0:strm] commitlog - Commitlog shutdown complete
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 0:strm] commitlog - Commitlog /var/lib/scylla/hints/0/51f9ec26-776f-4f66-96f0-51018af4e9b7 disposed
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 0:strm] commitlog - Commitlog shutdown complete
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 0:strm] commitlog - Commitlog /var/lib/scylla/hints/0/81ec2a79-17b5-451e-b6d1-960aab842f3f disposed
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 0:strm] commitlog - Commitlog shutdown complete
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 0:strm] commitlog - Commitlog /var/lib/scylla/hints/0/dcfb91cf-5d0a-436b-b71a-a554c1bd1bc5 disposed
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 0:strm] commitlog - Commitlog /var/lib/scylla/hints/0/dcfb91cf-5d0a-436b-b71a-a554c1bd1bc5 maximum disk size: 22 MB / cpu (6 cpus)
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 0:strm] commitlog - Commitlog /var/lib/scylla/hints/0/51f9ec26-776f-4f66-96f0-51018af4e9b7 maximum disk size: 22 MB / cpu (6 cpus)
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 0:strm] commitlog - Commitlog /var/lib/scylla/hints/0/81ec2a79-17b5-451e-b6d1-960aab842f3f maximum disk size: 22 MB / cpu (6 cpus)
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 0:strm] commitlog - Commitlog /var/lib/scylla/hints/0/17b6fc29-812b-4812-bc3a-ecfa256828f2 maximum disk size: 22 MB / cpu (6 cpus)
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 0:strm] commitlog - Delaying timer loop 0 ms
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 0:strm] commitlog - Delaying timer loop 0 ms
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 0:strm] commitlog - Delaying timer loop 0 ms
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 0:strm] commitlog - Delaying timer loop 0 ms
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 0:main] commitlog - Created new segment HintsLog-3-28417994.log
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 0:main] commitlog - Created new segment HintsLog-3-28417993.log
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 0:main] commitlog - Created new segment HintsLog-3-28418003.log
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 0:main] commitlog - Created new segment HintsLog-3-28418000.log
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 4:stmt] commitlog - Writing 249 entries, 131072 k in 4468736 -> 4599808
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 4:stmt] commitlog - Allocated 128 k buffer
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 4:clog] commitlog - Final write of 131072 to CommitLog-3-72057594066346605.log: 131072/131072 bytes at 4468736
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 4:strm] commitlog - Issuing sync for all segments
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 4:strm] commitlog - Checking for unused segments (0 active)
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 4:strm] commitlog - Issuing shutdown for all segments
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 4:strm] commitlog - Checking for unused segments (0 active)
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 4:strm] commitlog - Segment HintsLog-3-72057594066345939.log is no longer active and will submitted for delete now
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 4:strm] commitlog - Discarding segments [/var/lib/scylla/hints/4/81ec2a79-17b5-451e-b6d1-960aab842f3f/HintsLog-3-72057594066345939.log (33554432) (Force Delete)]
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 4:strm] commitlog - Issuing sync for all segments
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 4:strm] commitlog - Checking for unused segments (0 active)
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 4:strm] commitlog - Issuing shutdown for all segments
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 4:strm] commitlog - Checking for unused segments (0 active)
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 4:strm] commitlog - Segment HintsLog-3-72057594066345937.log is no longer active and will submitted for delete now
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 4:strm] commitlog - Discarding segments [/var/lib/scylla/hints/4/864ba0e1-5a86-4ac9-a1c2-0f75836c13f3/HintsLog-3-72057594066345937.log (33554432) (Force Delete)]
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 4:strm] commitlog - Issuing sync for all segments
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 4:strm] commitlog - Checking for unused segments (0 active)
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 4:strm] commitlog - Issuing shutdown for all segments
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 4:strm] commitlog - Checking for unused segments (0 active)
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 4:strm] commitlog - Segment HintsLog-3-72057594066345930.log is no longer active and will submitted for delete now
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 4:strm] commitlog - Discarding segments [/var/lib/scylla/hints/4/dcfb91cf-5d0a-436b-b71a-a554c1bd1bc5/HintsLog-3-72057594066345930.log (33554432) (Force Delete)]
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 4:strm] commitlog - Issuing sync for all segments
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 4:strm] commitlog - Checking for unused segments (0 active)
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 4:strm] commitlog - Issuing shutdown for all segments
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 4:strm] commitlog - Checking for unused segments (0 active)
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 4:strm] commitlog - Segment HintsLog-3-72057594066345929.log is no longer active and will submitted for delete now
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 4:strm] commitlog - Discarding segments [/var/lib/scylla/hints/4/51f9ec26-776f-4f66-96f0-51018af4e9b7/HintsLog-3-72057594066345929.log (33554432) (Force Delete)]
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 4:strm] commitlog - Deleting segment file /var/lib/scylla/hints/4/81ec2a79-17b5-451e-b6d1-960aab842f3f/HintsLog-3-72057594066345939.log
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 4:strm] commitlog - Deleting segment file /var/lib/scylla/hints/4/864ba0e1-5a86-4ac9-a1c2-0f75836c13f3/HintsLog-3-72057594066345937.log
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 4:strm] commitlog - Deleting segment file /var/lib/scylla/hints/4/dcfb91cf-5d0a-436b-b71a-a554c1bd1bc5/HintsLog-3-72057594066345930.log
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 4:strm] commitlog - Deleting segment file /var/lib/scylla/hints/4/51f9ec26-776f-4f66-96f0-51018af4e9b7/HintsLog-3-72057594066345929.log
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 4:strm] commitlog - Commitlog shutdown complete
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 4:strm] commitlog - Commitlog /var/lib/scylla/hints/4/81ec2a79-17b5-451e-b6d1-960aab842f3f disposed
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 4:strm] commitlog - Commitlog shutdown complete
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 4:strm] commitlog - Commitlog /var/lib/scylla/hints/4/864ba0e1-5a86-4ac9-a1c2-0f75836c13f3 disposed
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 4:strm] commitlog - Commitlog shutdown complete
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 4:strm] commitlog - Commitlog /var/lib/scylla/hints/4/dcfb91cf-5d0a-436b-b71a-a554c1bd1bc5 disposed
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 4:strm] commitlog - Commitlog shutdown complete
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 4:strm] commitlog - Commitlog /var/lib/scylla/hints/4/51f9ec26-776f-4f66-96f0-51018af4e9b7 disposed
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 4:strm] commitlog - Commitlog /var/lib/scylla/hints/4/81ec2a79-17b5-451e-b6d1-960aab842f3f maximum disk size: 22 MB / cpu (6 cpus)
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 4:strm] commitlog - Commitlog /var/lib/scylla/hints/4/dcfb91cf-5d0a-436b-b71a-a554c1bd1bc5 maximum disk size: 22 MB / cpu (6 cpus)
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 4:strm] commitlog - Commitlog /var/lib/scylla/hints/4/51f9ec26-776f-4f66-96f0-51018af4e9b7 maximum disk size: 22 MB / cpu (6 cpus)
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 4:strm] commitlog - Delaying timer loop 6668 ms
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 4:strm] commitlog - Delaying timer loop 6668 ms
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 4:strm] commitlog - Delaying timer loop 6668 ms
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 4:main] commitlog - Created new segment HintsLog-3-72057594066345939.log
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 4:main] commitlog - Created new segment HintsLog-3-72057594066345930.log
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 4:main] commitlog - Created new segment HintsLog-3-72057594066345929.log
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 4:strm] commitlog - Commitlog /var/lib/scylla/hints/4/864ba0e1-5a86-4ac9-a1c2-0f75836c13f3 maximum disk size: 22 MB / cpu (6 cpus)
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 4:strm] commitlog - Delaying timer loop 6668 ms
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 4:main] commitlog - Created new segment HintsLog-3-72057594066345937.log
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 0: gms] commitlog - Allocated 128 k buffer
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 0: gms] commitlog - Writing 1 entries, 4096 k in 122343424 -> 122347520
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 0: gms] commitlog - Allocated 128 k buffer
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 0:sclg] commitlog - Final write of 4096 to SchemaLog-3-28418171.log: 4096/4096 bytes at 122343424
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 0: gms] commitlog - SchemaLog-3-28418171.log synced to 122347520
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 0: gms] commitlog - Writing 5 entries, 4096 k in 122347520 -> 122351616
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 0:sclg] commitlog - Final write of 4096 to SchemaLog-3-28418171.log: 4096/4096 bytes at 122347520
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 0: gms] commitlog - SchemaLog-3-28418171.log synced to 122351616
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 1:strm] commitlog - Issuing sync for all segments
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 1:strm] commitlog - Issuing sync for all segments
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 1:strm] commitlog - Checking for unused segments (0 active)
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 1:strm] commitlog - Issuing shutdown for all segments
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 1:strm] commitlog - Checking for unused segments (0 active)
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 1:strm] commitlog - Segment HintsLog-3-18014398537899987.log is no longer active and will submitted for delete now
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 1:strm] commitlog - Discarding segments [/var/lib/scylla/hints/1/80361f2f-c12f-453e-8d08-01b19dcdf046/HintsLog-3-18014398537899987.log (33554432) (Force Delete)]
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 1:strm] commitlog - Checking for unused segments (0 active)
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 1:strm] commitlog - Issuing shutdown for all segments
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 1:strm] commitlog - Checking for unused segments (0 active)
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 1:strm] commitlog - Segment HintsLog-3-18014398537899988.log is no longer active and will submitted for delete now
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 1:strm] commitlog - Discarding segments [/var/lib/scylla/hints/1/81ec2a79-17b5-451e-b6d1-960aab842f3f/HintsLog-3-18014398537899988.log (33554432) (Force Delete)]
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 1:strm] commitlog - Issuing sync for all segments
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 1:strm] commitlog - Checking for unused segments (0 active)
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 1:strm] commitlog - Issuing shutdown for all segments
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 1:strm] commitlog - Checking for unused segments (0 active)
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 1:strm] commitlog - Segment HintsLog-3-18014398537899978.log is no longer active and will submitted for delete now
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 1:strm] commitlog - Discarding segments [/var/lib/scylla/hints/1/dcfb91cf-5d0a-436b-b71a-a554c1bd1bc5/HintsLog-3-18014398537899978.log (33554432) (Force Delete)]
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 1:strm] commitlog - Issuing sync for all segments
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 1:strm] commitlog - Checking for unused segments (0 active)
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 1:strm] commitlog - Issuing shutdown for all segments
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 1:strm] commitlog - Checking for unused segments (0 active)
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 1:strm] commitlog - Segment HintsLog-3-18014398537899977.log is no longer active and will submitted for delete now
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 1:strm] commitlog - Discarding segments [/var/lib/scylla/hints/1/51f9ec26-776f-4f66-96f0-51018af4e9b7/HintsLog-3-18014398537899977.log (33554432) (Force Delete)]
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 1:strm] commitlog - Deleting segment file /var/lib/scylla/hints/1/80361f2f-c12f-453e-8d08-01b19dcdf046/HintsLog-3-18014398537899987.log
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 1:strm] commitlog - Deleting segment file /var/lib/scylla/hints/1/81ec2a79-17b5-451e-b6d1-960aab842f3f/HintsLog-3-18014398537899988.log
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 1:strm] commitlog - Deleting segment file /var/lib/scylla/hints/1/dcfb91cf-5d0a-436b-b71a-a554c1bd1bc5/HintsLog-3-18014398537899978.log
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 1:strm] commitlog - Deleting segment file /var/lib/scylla/hints/1/51f9ec26-776f-4f66-96f0-51018af4e9b7/HintsLog-3-18014398537899977.log
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 1:strm] commitlog - Commitlog shutdown complete
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 1:strm] commitlog - Commitlog /var/lib/scylla/hints/1/80361f2f-c12f-453e-8d08-01b19dcdf046 disposed
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 1:strm] commitlog - Commitlog shutdown complete
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 1:strm] commitlog - Commitlog /var/lib/scylla/hints/1/81ec2a79-17b5-451e-b6d1-960aab842f3f disposed
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 1:strm] commitlog - Commitlog shutdown complete
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 1:strm] commitlog - Commitlog /var/lib/scylla/hints/1/dcfb91cf-5d0a-436b-b71a-a554c1bd1bc5 disposed
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 1:strm] commitlog - Commitlog shutdown complete
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 1:strm] commitlog - Commitlog /var/lib/scylla/hints/1/51f9ec26-776f-4f66-96f0-51018af4e9b7 disposed
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 1:strm] commitlog - Commitlog /var/lib/scylla/hints/1/51f9ec26-776f-4f66-96f0-51018af4e9b7 maximum disk size: 22 MB / cpu (6 cpus)
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 1:strm] commitlog - Commitlog /var/lib/scylla/hints/1/80361f2f-c12f-453e-8d08-01b19dcdf046 maximum disk size: 22 MB / cpu (6 cpus)
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 1:strm] commitlog - Commitlog /var/lib/scylla/hints/1/81ec2a79-17b5-451e-b6d1-960aab842f3f maximum disk size: 22 MB / cpu (6 cpus)
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 1:strm] commitlog - Delaying timer loop 1667 ms
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 1:strm] commitlog - Delaying timer loop 1667 ms
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 1:strm] commitlog - Delaying timer loop 1667 ms
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 1:main] commitlog - Created new segment HintsLog-3-18014398537899977.log
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 1:strm] commitlog - Commitlog /var/lib/scylla/hints/1/dcfb91cf-5d0a-436b-b71a-a554c1bd1bc5 maximum disk size: 22 MB / cpu (6 cpus)
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 1:main] commitlog - Created new segment HintsLog-3-18014398537899987.log
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 1:main] commitlog - Created new segment HintsLog-3-18014398537899988.log
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 1:strm] commitlog - Delaying timer loop 1667 ms
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 1:main] commitlog - Created new segment HintsLog-3-18014398537899978.log
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 2:strm] commitlog - Issuing sync for all segments
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 2:strm] commitlog - Checking for unused segments (0 active)
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 2:strm] commitlog - Issuing shutdown for all segments
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 2:strm] commitlog - Checking for unused segments (0 active)
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 2:strm] commitlog - Segment HintsLog-3-36028797047381968.log is no longer active and will submitted for delete now
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 2:strm] commitlog - Discarding segments [/var/lib/scylla/hints/2/17b6fc29-812b-4812-bc3a-ecfa256828f2/HintsLog-3-36028797047381968.log (33554432) (Force Delete)]
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 2:strm] commitlog - Issuing sync for all segments
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 2:strm] commitlog - Checking for unused segments (0 active)
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 2:strm] commitlog - Issuing shutdown for all segments
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 2:strm] commitlog - Checking for unused segments (0 active)
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 2:strm] commitlog - Segment HintsLog-3-36028797047381961.log is no longer active and will submitted for delete now
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 2:strm] commitlog - Discarding segments [/var/lib/scylla/hints/2/51f9ec26-776f-4f66-96f0-51018af4e9b7/HintsLog-3-36028797047381961.log (33554432) (Force Delete)]
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 2:strm] commitlog - Issuing sync for all segments
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 2:strm] commitlog - Checking for unused segments (0 active)
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 2:strm] commitlog - Issuing shutdown for all segments
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 2:strm] commitlog - Checking for unused segments (0 active)
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 2:strm] commitlog - Segment HintsLog-3-36028797047381961.log is no longer active and will submitted for delete now
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 2:strm] commitlog - Discarding segments [/var/lib/scylla/hints/2/dcfb91cf-5d0a-436b-b71a-a554c1bd1bc5/HintsLog-3-36028797047381961.log (33554432) (Force Delete)]
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 2:strm] commitlog - Issuing sync for all segments
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 2:strm] commitlog - Checking for unused segments (0 active)
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 2:strm] commitlog - Issuing shutdown for all segments
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 2:strm] commitlog - Checking for unused segments (0 active)
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 2:strm] commitlog - Segment HintsLog-3-36028797047381971.log is no longer active and will submitted for delete now
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 2:strm] commitlog - Discarding segments [/var/lib/scylla/hints/2/81ec2a79-17b5-451e-b6d1-960aab842f3f/HintsLog-3-36028797047381971.log (33554432) (Force Delete)]
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 2:strm] commitlog - Deleting segment file /var/lib/scylla/hints/2/17b6fc29-812b-4812-bc3a-ecfa256828f2/HintsLog-3-36028797047381968.log
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 2:strm] commitlog - Deleting segment file /var/lib/scylla/hints/2/51f9ec26-776f-4f66-96f0-51018af4e9b7/HintsLog-3-36028797047381961.log
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 2:strm] commitlog - Deleting segment file /var/lib/scylla/hints/2/dcfb91cf-5d0a-436b-b71a-a554c1bd1bc5/HintsLog-3-36028797047381961.log
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 2:strm] commitlog - Deleting segment file /var/lib/scylla/hints/2/81ec2a79-17b5-451e-b6d1-960aab842f3f/HintsLog-3-36028797047381971.log
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 2:strm] commitlog - Commitlog shutdown complete
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 2:strm] commitlog - Commitlog /var/lib/scylla/hints/2/17b6fc29-812b-4812-bc3a-ecfa256828f2 disposed
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 2:strm] commitlog - Commitlog shutdown complete
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 2:strm] commitlog - Commitlog /var/lib/scylla/hints/2/51f9ec26-776f-4f66-96f0-51018af4e9b7 disposed
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 2:strm] commitlog - Commitlog shutdown complete
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 2:strm] commitlog - Commitlog /var/lib/scylla/hints/2/dcfb91cf-5d0a-436b-b71a-a554c1bd1bc5 disposed
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 2:strm] commitlog - Commitlog shutdown complete
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 2:strm] commitlog - Commitlog /var/lib/scylla/hints/2/81ec2a79-17b5-451e-b6d1-960aab842f3f disposed
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 2:strm] commitlog - Issuing sync for all segments
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 2:strm] commitlog - Checking for unused segments (0 active)
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 2:strm] commitlog - Issuing shutdown for all segments
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 2:strm] commitlog - Checking for unused segments (0 active)
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 2:strm] commitlog - Segment HintsLog-3-36028797047381971.log is no longer active and will submitted for delete now
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 2:strm] commitlog - Discarding segments [/var/lib/scylla/hints/2/80361f2f-c12f-453e-8d08-01b19dcdf046/HintsLog-3-36028797047381971.log (33554432) (Force Delete)]
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 2:strm] commitlog - Deleting segment file /var/lib/scylla/hints/2/80361f2f-c12f-453e-8d08-01b19dcdf046/HintsLog-3-36028797047381971.log
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 2:strm] commitlog - Commitlog shutdown complete
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 2:strm] commitlog - Commitlog /var/lib/scylla/hints/2/80361f2f-c12f-453e-8d08-01b19dcdf046 disposed
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 2:strm] commitlog - Commitlog /var/lib/scylla/hints/2/81ec2a79-17b5-451e-b6d1-960aab842f3f maximum disk size: 22 MB / cpu (6 cpus)
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 2:strm] commitlog - Commitlog /var/lib/scylla/hints/2/17b6fc29-812b-4812-bc3a-ecfa256828f2 maximum disk size: 22 MB / cpu (6 cpus)
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 2:strm] commitlog - Commitlog /var/lib/scylla/hints/2/51f9ec26-776f-4f66-96f0-51018af4e9b7 maximum disk size: 22 MB / cpu (6 cpus)
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 2:strm] commitlog - Delaying timer loop 3334 ms
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 2:strm] commitlog - Delaying timer loop 3334 ms
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 2:strm] commitlog - Delaying timer loop 3334 ms
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 2:main] commitlog - Created new segment HintsLog-3-36028797047381971.log
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 2:main] commitlog - Created new segment HintsLog-3-36028797047381968.log
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 2:strm] commitlog - Commitlog /var/lib/scylla/hints/2/dcfb91cf-5d0a-436b-b71a-a554c1bd1bc5 maximum disk size: 22 MB / cpu (6 cpus)
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 2:strm] commitlog - Commitlog /var/lib/scylla/hints/2/80361f2f-c12f-453e-8d08-01b19dcdf046 maximum disk size: 22 MB / cpu (6 cpus)
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 2:main] commitlog - Created new segment HintsLog-3-36028797047381961.log
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 2:strm] commitlog - Delaying timer loop 3334 ms
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 2:strm] commitlog - Delaying timer loop 3334 ms
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 2:main] commitlog - Created new segment HintsLog-3-36028797047381961.log
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 2:main] commitlog - Created new segment HintsLog-3-36028797047381971.log
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 5:strm] commitlog - Issuing sync for all segments
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 5:strm] commitlog - Checking for unused segments (0 active)
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 5:strm] commitlog - Issuing shutdown for all segments
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 5:strm] commitlog - Checking for unused segments (0 active)
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 5:strm] commitlog - Segment HintsLog-3-90071992575827924.log is no longer active and will submitted for delete now
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 5:strm] commitlog - Discarding segments [/var/lib/scylla/hints/5/80361f2f-c12f-453e-8d08-01b19dcdf046/HintsLog-3-90071992575827924.log (33554432) (Force Delete)]
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 5:strm] commitlog - Issuing sync for all segments
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 5:strm] commitlog - Checking for unused segments (0 active)
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 5:strm] commitlog - Issuing shutdown for all segments
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 5:strm] commitlog - Checking for unused segments (0 active)
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 5:strm] commitlog - Segment HintsLog-3-90071992575827923.log is no longer active and will submitted for delete now
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 5:strm] commitlog - Discarding segments [/var/lib/scylla/hints/5/81ec2a79-17b5-451e-b6d1-960aab842f3f/HintsLog-3-90071992575827923.log (33554432) (Force Delete)]
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 5:strm] commitlog - Issuing sync for all segments
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 5:strm] commitlog - Checking for unused segments (0 active)
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 5:strm] commitlog - Issuing shutdown for all segments
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 5:strm] commitlog - Checking for unused segments (0 active)
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 5:strm] commitlog - Segment HintsLog-3-90071992575827920.log is no longer active and will submitted for delete now
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 5:strm] commitlog - Discarding segments [/var/lib/scylla/hints/5/17b6fc29-812b-4812-bc3a-ecfa256828f2/HintsLog-3-90071992575827920.log (33554432) (Force Delete)]
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 5:strm] commitlog - Issuing sync for all segments
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 5:strm] commitlog - Checking for unused segments (0 active)
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 5:strm] commitlog - Issuing shutdown for all segments
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 5:strm] commitlog - Checking for unused segments (0 active)
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 5:strm] commitlog - Segment HintsLog-3-90071992575827913.log is no longer active and will submitted for delete now
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 5:strm] commitlog - Discarding segments [/var/lib/scylla/hints/5/51f9ec26-776f-4f66-96f0-51018af4e9b7/HintsLog-3-90071992575827913.log (33554432) (Force Delete)]
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 5:strm] commitlog - Issuing sync for all segments
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 5:strm] commitlog - Checking for unused segments (0 active)
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 5:strm] commitlog - Issuing shutdown for all segments
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 5:strm] commitlog - Checking for unused segments (0 active)
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 5:strm] commitlog - Segment HintsLog-3-90071992575827914.log is no longer active and will submitted for delete now
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 5:strm] commitlog - Discarding segments [/var/lib/scylla/hints/5/dcfb91cf-5d0a-436b-b71a-a554c1bd1bc5/HintsLog-3-90071992575827914.log (33554432) (Force Delete)]
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 5:strm] commitlog - Deleting segment file /var/lib/scylla/hints/5/80361f2f-c12f-453e-8d08-01b19dcdf046/HintsLog-3-90071992575827924.log
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 5:strm] commitlog - Deleting segment file /var/lib/scylla/hints/5/81ec2a79-17b5-451e-b6d1-960aab842f3f/HintsLog-3-90071992575827923.log
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 5:strm] commitlog - Deleting segment file /var/lib/scylla/hints/5/17b6fc29-812b-4812-bc3a-ecfa256828f2/HintsLog-3-90071992575827920.log
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 5:strm] commitlog - Deleting segment file /var/lib/scylla/hints/5/51f9ec26-776f-4f66-96f0-51018af4e9b7/HintsLog-3-90071992575827913.log
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 5:strm] commitlog - Deleting segment file /var/lib/scylla/hints/5/dcfb91cf-5d0a-436b-b71a-a554c1bd1bc5/HintsLog-3-90071992575827914.log
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 5:strm] commitlog - Commitlog shutdown complete
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 5:strm] commitlog - Commitlog /var/lib/scylla/hints/5/80361f2f-c12f-453e-8d08-01b19dcdf046 disposed
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 5:strm] commitlog - Commitlog shutdown complete
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 5:strm] commitlog - Commitlog /var/lib/scylla/hints/5/81ec2a79-17b5-451e-b6d1-960aab842f3f disposed
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 5:strm] commitlog - Commitlog shutdown complete
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 5:strm] commitlog - Commitlog /var/lib/scylla/hints/5/17b6fc29-812b-4812-bc3a-ecfa256828f2 disposed
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 5:strm] commitlog - Commitlog shutdown complete
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 5:strm] commitlog - Commitlog /var/lib/scylla/hints/5/51f9ec26-776f-4f66-96f0-51018af4e9b7 disposed
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 5:strm] commitlog - Commitlog shutdown complete
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 5:strm] commitlog - Commitlog /var/lib/scylla/hints/5/dcfb91cf-5d0a-436b-b71a-a554c1bd1bc5 disposed
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 5:strm] commitlog - Commitlog /var/lib/scylla/hints/5/51f9ec26-776f-4f66-96f0-51018af4e9b7 maximum disk size: 22 MB / cpu (6 cpus)
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 5:strm] commitlog - Commitlog /var/lib/scylla/hints/5/81ec2a79-17b5-451e-b6d1-960aab842f3f maximum disk size: 22 MB / cpu (6 cpus)
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 5:strm] commitlog - Commitlog /var/lib/scylla/hints/5/dcfb91cf-5d0a-436b-b71a-a554c1bd1bc5 maximum disk size: 22 MB / cpu (6 cpus)
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 5:strm] commitlog - Commitlog /var/lib/scylla/hints/5/80361f2f-c12f-453e-8d08-01b19dcdf046 maximum disk size: 22 MB / cpu (6 cpus)
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 5:strm] commitlog - Commitlog /var/lib/scylla/hints/5/17b6fc29-812b-4812-bc3a-ecfa256828f2 maximum disk size: 22 MB / cpu (6 cpus)
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 5:strm] commitlog - Delaying timer loop 8335 ms
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 5:strm] commitlog - Delaying timer loop 8335 ms
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 5:strm] commitlog - Delaying timer loop 8335 ms
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 5:strm] commitlog - Delaying timer loop 8335 ms
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 5:strm] commitlog - Delaying timer loop 8335 ms
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 5:main] commitlog - Created new segment HintsLog-3-90071992575827913.log
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 5:main] commitlog - Created new segment HintsLog-3-90071992575827923.log
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 5:main] commitlog - Created new segment HintsLog-3-90071992575827914.log
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 5:main] commitlog - Created new segment HintsLog-3-90071992575827924.log
Oct 07 12:02:50 o-p-L3-8 scylla[32999]:  [shard 5:main] commitlog - Created new segment HintsLog-3-90071992575827920.log

avikivity · October 7, 2024, 1:59pm

Please migrate this to an issue, this is not a good place to debug.

horschi · October 7, 2024, 8:27pm

github.com/scylladb/scylladb

Schema commitlog continuesly updated

opened 08:24PM - 07 Oct 24 UTC

horschi

As discussed in: https://forum.scylladb.com/t/compaction-storm-slows-down-scylla…/2958/24 It seems Scylla 6.0.2 gets stuck in some loop after upgrade from 5.4 or when bootstrapping a new 6.0 node into a cluster. After enabled commitlog trace we can see: ``` Oct 07 12:02:47 o-p-L3-8 scylla[32999]: [shard 2:strm] commitlog - Segment HintsLog-3-36028797047381970.log is no longer active and will submitted for delete now Oct 07 12:02:47 o-p-L3-8 scylla[32999]: [shard 2:strm] commitlog - Discarding segments [/var/lib/scylla/hints/2/d01ea9d0-787f-4aa5-9235-6f9b3d424063/HintsLog-3-36028797047381970.log (33554432) (Force Delete)] Oct 07 12:02:47 o-p-L3-8 scylla[32999]: [shard 2:strm] commitlog - Deleting segment file /var/lib/scylla/hints/2/d01ea9d0-787f-4aa5-9235-6f9b3d424063/HintsLog-3-36028797047381970.log Oct 07 12:02:47 o-p-L3-8 scylla[32999]: [shard 2:strm] commitlog - Commitlog shutdown complete Oct 07 12:02:47 o-p-L3-8 scylla[32999]: [shard 2:strm] commitlog - Commitlog /var/lib/scylla/hints/2/d01ea9d0-787f-4aa5-9235-6f9b3d424063 disposed Oct 07 12:02:47 o-p-L3-8 scylla[32999]: [shard 2:strm] commitlog - Commitlog /var/lib/scylla/hints/2/d01ea9d0-787f-4aa5-9235-6f9b3d424063 maximum disk size: 22 MB / cpu (6 cpus) Oct 07 12:02:47 o-p-L3-8 scylla[32999]: [shard 2:strm] commitlog - Delaying timer loop 3334 ms Oct 07 12:02:47 o-p-L3-8 scylla[32999]: [shard 2:main] commitlog - Created new segment HintsLog-3-36028797047381970.log Oct 07 12:02:47 o-p-L3-8 scylla[32999]: [shard 0: gms] commitlog - Allocated 128 k buffer Oct 07 12:02:47 o-p-L3-8 scylla[32999]: [shard 0: gms] commitlog - Writing 1 entries, 4096 k in 122220544 -> 122224640 Oct 07 12:02:47 o-p-L3-8 scylla[32999]: [shard 0: gms] commitlog - Allocated 128 k buffer Oct 07 12:02:47 o-p-L3-8 scylla[32999]: [shard 0:sclg] commitlog - Final write of 4096 to SchemaLog-3-28418171.log: 4096/4096 bytes at 122220544 Oct 07 12:02:47 o-p-L3-8 scylla[32999]: [shard 0: gms] commitlog - SchemaLog-3-28418171.log synced to 122224640 Oct 07 12:02:47 o-p-L3-8 scylla[32999]: [shard 0: gms] commitlog - Writing 1 entries, 4096 k in 122224640 -> 122228736 Oct 07 12:02:47 o-p-L3-8 scylla[32999]: [shard 0:sclg] commitlog - Final write of 4096 to SchemaLog-3-28418171.log: 4096/4096 bytes at 122224640 Oct 07 12:02:47 o-p-L3-8 scylla[32999]: [shard 0: gms] commitlog - SchemaLog-3-28418171.log synced to 122228736 Oct 07 12:02:47 o-p-L3-8 scylla[32999]: [shard 0: gms] commitlog - Allocated 128 k buffer Oct 07 12:02:47 o-p-L3-8 scylla[32999]: [shard 0: gms] commitlog - Writing 1 entries, 4096 k in 122228736 -> 122232832 Oct 07 12:02:47 o-p-L3-8 scylla[32999]: [shard 0: gms] commitlog - Allocated 128 k buffer Oct 07 12:02:47 o-p-L3-8 scylla[32999]: [shard 0:sclg] commitlog - Final write of 4096 to SchemaLog-3-28418171.log: 4096/4096 bytes at 122228736 Oct 07 12:02:47 o-p-L3-8 scylla[32999]: [shard 0: gms] commitlog - SchemaLog-3-28418171.log synced to 122232832 Oct 07 12:02:47 o-p-L3-8 scylla[32999]: [shard 0: gms] commitlog - Writing 9 entries, 4096 k in 122232832 -> 122236928 Oct 07 12:02:47 o-p-L3-8 scylla[32999]: [shard 0:sclg] commitlog - Final write of 4096 to SchemaLog-3-28418171.log: 4096/4096 bytes at 122232832 Oct 07 12:02:47 o-p-L3-8 scylla[32999]: [shard 0: gms] commitlog - SchemaLog-3-28418171.log synced to 122236928 Oct 07 12:02:47 o-p-L3-8 scylla[32999]: [shard 1:stmt] commitlog - Writing 302 entries, 131072 k in 5058560 -> 5189632 Oct 07 12:02:47 o-p-L3-8 scylla[32999]: [shard 1:stmt] commitlog - Allocated 128 k buffer Oct 07 12:02:47 o-p-L3-8 scylla[32999]: [shard 1:clog] commitlog - Final write of 131072 to CommitLog-3-18014398537900605.log: 131072/131072 bytes at 5058560 Oct 07 12:02:47 o-p-L3-8 scylla[32999]: [shard 3:stmt] commitlog - Writing 258 entries, 131072 k in 28348416 -> 28479488 Oct 07 12:02:47 o-p-L3-8 scylla[32999]: [shard 3:stmt] commitlog - Allocated 128 k buffer Oct 07 12:02:47 o-p-L3-8 scylla[32999]: [shard 3:clog] commitlog - Final write of 131072 to CommitLog-3-54043195556864646.log: 131072/131072 bytes at 28348416 Oct 07 12:02:47 o-p-L3-8 scylla[32999]: [shard 0: gms] commitlog - Allocated 128 k buffer Oct 07 12:02:47 o-p-L3-8 scylla[32999]: [shard 0: gms] commitlog - Writing 1 entries, 4096 k in 122236928 -> 122241024 Oct 07 12:02:47 o-p-L3-8 scylla[32999]: [shard 0: gms] commitlog - Allocated 128 k buffer Oct 07 12:02:47 o-p-L3-8 scylla[32999]: [shard 0:sclg] commitlog - Final write of 4096 to SchemaLog-3-28418171.log: 4096/4096 bytes at 122236928 Oct 07 12:02:47 o-p-L3-8 scylla[32999]: [shard 0: gms] commitlog - SchemaLog-3-28418171.log synced to 122241024 Oct 07 12:02:47 o-p-L3-8 scylla[32999]: [shard 0: gms] commitlog - Writing 4 entries, 4096 k in 122241024 -> 122245120 Oct 07 12:02:47 o-p-L3-8 scylla[32999]: [shard 0:sclg] commitlog - Final write of 4096 to SchemaLog-3-28418171.log: 4096/4096 bytes at 122241024 Oct 07 12:02:47 o-p-L3-8 scylla[32999]: [shard 0: gms] commitlog - SchemaLog-3-28418171.log synced to 122245120 Oct 07 12:02:47 o-p-L3-8 scylla[32999]: [shard 5:stmt] commitlog - Writing 360 entries, 131072 k in 34680832 -> 34811904 Oct 07 12:02:47 o-p-L3-8 scylla[32999]: [shard 5:stmt] commitlog - Allocated 128 k buffer Oct 07 12:02:47 o-p-L3-8 scylla[32999]: [shard 5:clog] commitlog - Final write of 131072 to CommitLog-3-90071992575828770.log: 131072/131072 bytes at 34680832 Oct 07 12:02:47 o-p-L3-8 scylla[32999]: [shard 0: gms] commitlog - Allocated 128 k buffer Oct 07 12:02:47 o-p-L3-8 scylla[32999]: [shard 0: gms] commitlog - Writing 1 entries, 4096 k in 122245120 -> 122249216 Oct 07 12:02:47 o-p-L3-8 scylla[32999]: [shard 0: gms] commitlog - Allocated 128 k buffer Oct 07 12:02:47 o-p-L3-8 scylla[32999]: [shard 0:sclg] commitlog - Final write of 4096 to SchemaLog-3-28418171.log: 4096/4096 bytes at 122245120 Oct 07 12:02:47 o-p-L3-8 scylla[32999]: [shard 0: gms] commitlog - SchemaLog-3-28418171.log synced to 122249216 Oct 07 12:02:47 o-p-L3-8 scylla[32999]: [shard 0: gms] commitlog - Writing 4 entries, 4096 k in 122249216 -> 122253312 Oct 07 12:02:47 o-p-L3-8 scylla[32999]: [shard 0:sclg] commitlog - Final write of 4096 to SchemaLog-3-28418171.log: 4096/4096 bytes at 122249216 Oct 07 12:02:47 o-p-L3-8 scylla[32999]: [shard 0: gms] commitlog - SchemaLog-3-28418171.log synced to 122253312 Oct 07 12:02:47 o-p-L3-8 scylla[32999]: [shard 0: gms] commitlog - Allocated 128 k buffer Oct 07 12:02:47 o-p-L3-8 scylla[32999]: [shard 0: gms] commitlog - Writing 1 entries, 4096 k in 122253312 -> 122257408 Oct 07 12:02:47 o-p-L3-8 scylla[32999]: [shard 0: gms] commitlog - Allocated 128 k buffer Oct 07 12:02:47 o-p-L3-8 scylla[32999]: [shard 0:sclg] commitlog - Final write of 4096 to SchemaLog-3-28418171.log: 4096/4096 bytes at 122253312 Oct 07 12:02:47 o-p-L3-8 scylla[32999]: [shard 0: gms] commitlog - SchemaLog-3-28418171.log synced to 122257408 Oct 07 12:02:47 o-p-L3-8 scylla[32999]: [shard 0: gms] commitlog - Writing 10 entries, 4096 k in 122257408 -> 122261504 Oct 07 12:02:47 o-p-L3-8 scylla[32999]: [shard 0:sclg] commitlog - Final write of 4096 to SchemaLog-3-28418171.log: 4096/4096 bytes at 122257408 Oct 07 12:02:47 o-p-L3-8 scylla[32999]: [shard 0: gms] commitlog - SchemaLog-3-28418171.log synced to 122261504 Oct 07 12:02:47 o-p-L3-8 scylla[32999]: [shard 0: gms] commitlog - Allocated 128 k buffer Oct 07 12:02:47 o-p-L3-8 scylla[32999]: [shard 0: gms] commitlog - Writing 1 entries, 4096 k in 122261504 -> 122265600 Oct 07 12:02:47 o-p-L3-8 scylla[32999]: [shard 0: gms] commitlog - Allocated 128 k buffer Oct 07 12:02:47 o-p-L3-8 scylla[32999]: [shard 0:sclg] commitlog - Final write of 4096 to SchemaLog-3-28418171.log: 4096/4096 bytes at 122261504 Oct 07 12:02:47 o-p-L3-8 scylla[32999]: [shard 0: gms] commitlog - SchemaLog-3-28418171.log synced to 122265600 Oct 07 12:02:47 o-p-L3-8 scylla[32999]: [shard 0: gms] commitlog - Writing 2 entries, 4096 k in 122265600 -> 122269696 Oct 07 12:02:47 o-p-L3-8 scylla[32999]: [shard 0:sclg] commitlog - Final write of 4096 to SchemaLog-3-28418171.log: 4096/4096 bytes at 122265600 Oct 07 12:02:47 o-p-L3-8 scylla[32999]: [shard 0: gms] commitlog - SchemaLog-3-28418171.log synced to 122269696 Oct 07 12:02:48 o-p-L3-8 scylla[32999]: [shard 4:stmt] commitlog - Writing 234 entries, 102400 k in 4366336 -> 4468736 Oct 07 12:02:48 o-p-L3-8 scylla[32999]: [shard 4:stmt] commitlog - Allocated 128 k buffer Oct 07 12:02:48 o-p-L3-8 scylla[32999]: [shard 4:clog] commitlog - Final write of 102400 to CommitLog-3-72057594066346605.log: 102400/102400 bytes at 4366336 Oct 07 12:02:48 o-p-L3-8 scylla[32999]: [shard 2:stmt] commitlog - Writing 307 entries, 131072 k in 32587776 -> 32718848 Oct 07 12:02:48 o-p-L3-8 scylla[32999]: [shard 2:stmt] commitlog - Allocated 128 k buffer Oct 07 12:02:48 o-p-L3-8 scylla[32999]: [shard 2:clog] commitlog - Final write of 131072 to CommitLog-3-36028797047382674.log: 131072/131072 bytes at 32587776 Oct 07 12:02:48 o-p-L3-8 scylla[32999]: [shard 0: gms] commitlog - Allocated 128 k buffer Oct 07 12:02:48 o-p-L3-8 scylla[32999]: [shard 0: gms] commitlog - Writing 1 entries, 4096 k in 122269696 -> 122273792 Oct 07 12:02:48 o-p-L3-8 scylla[32999]: [shard 0: gms] commitlog - Allocated 128 k buffer Oct 07 12:02:48 o-p-L3-8 scylla[32999]: [shard 0:sclg] commitlog - Final write of 4096 to SchemaLog-3-28418171.log: 4096/4096 bytes at 122269696 Oct 07 12:02:48 o-p-L3-8 scylla[32999]: [shard 0: gms] commitlog - SchemaLog-3-28418171.log synced to 122273792 Oct 07 12:02:48 o-p-L3-8 scylla[32999]: [shard 0: gms] commitlog - Writing 25 entries, 8192 k in 122273792 -> 122281984 Oct 07 12:02:48 o-p-L3-8 scylla[32999]: [shard 0:sclg] commitlog - Final write of 8192 to SchemaLog-3-28418171.log: 8192/8192 bytes at 122273792 Oct 07 12:02:48 o-p-L3-8 scylla[32999]: [shard 0: gms] commitlog - SchemaLog-3-28418171.log synced to 122281984 Oct 07 12:02:48 o-p-L3-8 scylla[32999]: [shard 0: gms] commitlog - Allocated 128 k buffer Oct 07 12:02:48 o-p-L3-8 scylla[32999]: [shard 0: gms] commitlog - Writing 1 entries, 4096 k in 122281984 -> 122286080 Oct 07 12:02:48 o-p-L3-8 scylla[32999]: [shard 0: gms] commitlog - Allocated 128 k buffer Oct 07 12:02:48 o-p-L3-8 scylla[32999]: [shard 0:sclg] commitlog - Final write of 4096 to SchemaLog-3-28418171.log: 4096/4096 bytes at 122281984 Oct 07 12:02:48 o-p-L3-8 scylla[32999]: [shard 0: gms] commitlog - SchemaLog-3-28418171.log synced to 122286080 Oct 07 12:02:48 o-p-L3-8 scylla[32999]: [shard 0: gms] commitlog - Writing 2 entries, 4096 k in 122286080 -> 122290176 Oct 07 12:02:48 o-p-L3-8 scylla[32999]: [shard 0:sclg] commitlog - Final write of 4096 to SchemaLog-3-28418171.log: 4096/4096 bytes at 122286080 Oct 07 12:02:48 o-p-L3-8 scylla[32999]: [shard 0: gms] commitlog - SchemaLog-3-28418171.log synced to 122290176 Oct 07 12:02:48 o-p-L3-8 scylla[32999]: [shard 0: gms] commitlog - Allocated 128 k buffer Oct 07 12:02:48 o-p-L3-8 scylla[32999]: [shard 0: gms] commitlog - Writing 1 entries, 4096 k in 122290176 -> 122294272 Oct 07 12:02:48 o-p-L3-8 scylla[32999]: [shard 0:sclg] commitlog - Final write of 4096 to SchemaLog-3-28418171.log: 4096/4096 bytes at 122290176 Oct 07 12:02:48 o-p-L3-8 scylla[32999]: [shard 0: gms] commitlog - SchemaLog-3-28418171.log synced to 122294272 Oct 07 12:02:48 o-p-L3-8 scylla[32999]: [shard 1:stmt] commitlog - Writing 217 entries, 131072 k in 5189632 -> 5320704 Oct 07 12:02:48 o-p-L3-8 scylla[32999]: [shard 1:stmt] commitlog - Allocated 128 k buffer Oct 07 12:02:48 o-p-L3-8 scylla[32999]: [shard 1:clog] commitlog - Final write of 131072 to CommitLog-3-18014398537900605.log: 131072/131072 bytes at 5189632 Oct 07 12:02:49 o-p-L3-8 scylla[32999]: [shard 0: gms] commitlog - Allocated 128 k buffer Oct 07 12:02:49 o-p-L3-8 scylla[32999]: [shard 0: gms] commitlog - Writing 1 entries, 4096 k in 122294272 -> 122298368 Oct 07 12:02:49 o-p-L3-8 scylla[32999]: [shard 0: gms] commitlog - Allocated 128 k buffer Oct 07 12:02:49 o-p-L3-8 scylla[32999]: [shard 0:sclg] commitlog - Final write of 4096 to SchemaLog-3-28418171.log: 4096/4096 bytes at 122294272 Oct 07 12:02:49 o-p-L3-8 scylla[32999]: [shard 0: gms] commitlog - SchemaLog-3-28418171.log synced to 122298368 Oct 07 12:02:49 o-p-L3-8 scylla[32999]: [shard 0: gms] commitlog - Writing 1 entries, 4096 k in 122298368 -> 122302464 Oct 07 12:02:49 o-p-L3-8 scylla[32999]: [shard 0:sclg] commitlog - Final write of 4096 to SchemaLog-3-28418171.log: 4096/4096 bytes at 122298368 Oct 07 12:02:49 o-p-L3-8 scylla[32999]: [shard 0: gms] commitlog - SchemaLog-3-28418171.log synced to 122302464 Oct 07 12:02:49 o-p-L3-8 scylla[32999]: [shard 0: gms] commitlog - Allocated 128 k buffer Oct 07 12:02:49 o-p-L3-8 scylla[32999]: [shard 0: gms] commitlog - Writing 1 entries, 4096 k in 122302464 -> 122306560 Oct 07 12:02:49 o-p-L3-8 scylla[32999]: [shard 0:sclg] commitlog - Final write of 4096 to SchemaLog-3-28418171.log: 4096/4096 bytes at 122302464 Oct 07 12:02:49 o-p-L3-8 scylla[32999]: [shard 0: gms] commitlog - SchemaLog-3-28418171.log synced to 122306560 Oct 07 12:02:49 o-p-L3-8 scylla[32999]: [shard 0:stmt] commitlog - Writing 376 entries, 131072 k in 63127552 -> 63258624 Oct 07 12:02:49 o-p-L3-8 scylla[32999]: [shard 0:stmt] commitlog - Allocated 128 k buffer Oct 07 12:02:49 o-p-L3-8 scylla[32999]: [shard 0:clog] commitlog - Final write of 131072 to CommitLog-3-28418828.log: 131072/131072 bytes at 63127552 Oct 07 12:02:49 o-p-L3-8 scylla[32999]: [shard 0: gms] commitlog - Allocated 128 k buffer Oct 07 12:02:49 o-p-L3-8 scylla[32999]: [shard 0: gms] commitlog - Writing 1 entries, 4096 k in 122306560 -> 122310656 Oct 07 12:02:49 o-p-L3-8 scylla[32999]: [shard 0: gms] commitlog - Allocated 128 k buffer Oct 07 12:02:49 o-p-L3-8 scylla[32999]: [shard 0:sclg] commitlog - Final write of 4096 to SchemaLog-3-28418171.log: 4096/4096 bytes at 122306560 Oct 07 12:02:49 o-p-L3-8 scylla[32999]: [shard 0: gms] commitlog - SchemaLog-3-28418171.log synced to 122310656 Oct 07 12:02:49 o-p-L3-8 scylla[32999]: [shard 0: gms] commitlog - Writing 3 entries, 4096 k in 122310656 -> 122314752 Oct 07 12:02:49 o-p-L3-8 scylla[32999]: [shard 0:sclg] commitlog - Final write of 4096 to SchemaLog-3-28418171.log: 4096/4096 bytes at 122310656 Oct 07 12:02:49 o-p-L3-8 scylla[32999]: [shard 0: gms] commitlog - SchemaLog-3-28418171.log synced to 122314752 Oct 07 12:02:49 o-p-L3-8 scylla[32999]: [shard 3:stmt] commitlog - Writing 252 entries, 131072 k in 28479488 -> 28610560 Oct 07 12:02:49 o-p-L3-8 scylla[32999]: [shard 3:stmt] commitlog - Allocated 128 k buffer Oct 07 12:02:49 o-p-L3-8 scylla[32999]: [shard 3:clog] commitlog - Final write of 131072 to CommitLog-3-54043195556864646.log: 131072/131072 bytes at 28479488 Oct 07 12:02:49 o-p-L3-8 scylla[32999]: [shard 5:stmt] commitlog - Writing 293 entries, 131072 k in 34811904 -> 34942976 Oct 07 12:02:49 o-p-L3-8 scylla[32999]: [shard 5:stmt] commitlog - Allocated 128 k buffer Oct 07 12:02:49 o-p-L3-8 scylla[32999]: [shard 5:clog] commitlog - Final write of 131072 to CommitLog-3-90071992575828770.log: 131072/131072 bytes at 34811904 Oct 07 12:02:49 o-p-L3-8 scylla[32999]: [shard 0: gms] commitlog - Allocated 128 k buffer Oct 07 12:02:49 o-p-L3-8 scylla[32999]: [shard 0: gms] commitlog - Writing 1 entries, 4096 k in 122314752 -> 122318848 Oct 07 12:02:49 o-p-L3-8 scylla[32999]: [shard 0: gms] commitlog - Allocated 128 k buffer Oct 07 12:02:49 o-p-L3-8 scylla[32999]: [shard 0:sclg] commitlog - Final write of 4096 to SchemaLog-3-28418171.log: 4096/4096 bytes at 122314752 Oct 07 12:02:49 o-p-L3-8 scylla[32999]: [shard 0: gms] commitlog - SchemaLog-3-28418171.log synced to 122318848 Oct 07 12:02:49 o-p-L3-8 scylla[32999]: [shard 0: gms] commitlog - Writing 12 entries, 4096 k in 122318848 -> 122322944 Oct 07 12:02:49 o-p-L3-8 scylla[32999]: [shard 0:sclg] commitlog - Final write of 4096 to SchemaLog-3-28418171.log: 4096/4096 bytes at 122318848 Oct 07 12:02:49 o-p-L3-8 scylla[32999]: [shard 0: gms] commitlog - SchemaLog-3-28418171.log synced to 122322944 Oct 07 12:02:49 o-p-L3-8 scylla[32999]: [shard 0: gms] commitlog - Allocated 128 k buffer Oct 07 12:02:49 o-p-L3-8 scylla[32999]: [shard 0: gms] commitlog - Writing 1 entries, 4096 k in 122322944 -> 122327040 Oct 07 12:02:49 o-p-L3-8 scylla[32999]: [shard 0: gms] commitlog - Allocated 128 k buffer Oct 07 12:02:49 o-p-L3-8 scylla[32999]: [shard 0:sclg] commitlog - Final write of 4096 to SchemaLog-3-28418171.log: 4096/4096 bytes at 122322944 Oct 07 12:02:49 o-p-L3-8 scylla[32999]: [shard 0: gms] commitlog - SchemaLog-3-28418171.log synced to 122327040 Oct 07 12:02:49 o-p-L3-8 scylla[32999]: [shard 0: gms] commitlog - Writing 1 entries, 4096 k in 122327040 -> 122331136 Oct 07 12:02:49 o-p-L3-8 scylla[32999]: [shard 0: gms] commitlog - Allocated 128 k buffer Oct 07 12:02:49 o-p-L3-8 scylla[32999]: [shard 0:sclg] commitlog - Final write of 4096 to SchemaLog-3-28418171.log: 4096/4096 bytes at 122327040 Oct 07 12:02:49 o-p-L3-8 scylla[32999]: [shard 0: gms] commitlog - SchemaLog-3-28418171.log synced to 122331136 Oct 07 12:02:49 o-p-L3-8 scylla[32999]: [shard 0: gms] commitlog - Writing 10 entries, 4096 k in 122331136 -> 122335232 Oct 07 12:02:49 o-p-L3-8 scylla[32999]: [shard 0: gms] commitlog - Allocated 128 k buffer Oct 07 12:02:49 o-p-L3-8 scylla[32999]: [shard 0:sclg] commitlog - Final write of 4096 to SchemaLog-3-28418171.log: 4096/4096 bytes at 122331136 Oct 07 12:02:49 o-p-L3-8 scylla[32999]: [shard 0: gms] commitlog - SchemaLog-3-28418171.log synced to 122335232 Oct 07 12:02:49 o-p-L3-8 scylla[32999]: [shard 0: gms] commitlog - Writing 4 entries, 4096 k in 122335232 -> 122339328 Oct 07 12:02:49 o-p-L3-8 scylla[32999]: [shard 0: gms] commitlog - Allocated 128 k buffer Oct 07 12:02:49 o-p-L3-8 scylla[32999]: [shard 0:sclg] commitlog - Final write of 4096 to SchemaLog-3-28418171.log: 4096/4096 bytes at 122335232 Oct 07 12:02:49 o-p-L3-8 scylla[32999]: [shard 0: gms] commitlog - SchemaLog-3-28418171.log synced to 122339328 Oct 07 12:02:49 o-p-L3-8 scylla[32999]: [shard 0: gms] commitlog - Writing 1 entries, 4096 k in 122339328 -> 122343424 Oct 07 12:02:49 o-p-L3-8 scylla[32999]: [shard 0:sclg] commitlog - Final write of 4096 to SchemaLog-3-28418171.log: 4096/4096 bytes at 122339328 Oct 07 12:02:49 o-p-L3-8 scylla[32999]: [shard 0: gms] commitlog - SchemaLog-3-28418171.log synced to 122343424 Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 0:strm] commitlog - Issuing sync for all segments Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 0:strm] commitlog - Checking for unused segments (0 active) Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 0:strm] commitlog - Issuing shutdown for all segments Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 0:strm] commitlog - Checking for unused segments (0 active) Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 0:strm] commitlog - Segment HintsLog-3-28418000.log is no longer active and will submitted for delete now Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 0:strm] commitlog - Discarding segments [/var/lib/scylla/hints/0/17b6fc29-812b-4812-bc3a-ecfa256828f2/HintsLog-3-28418000.log (33554432) (Force Delete)] Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 0:strm] commitlog - Issuing sync for all segments Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 0:strm] commitlog - Checking for unused segments (0 active) Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 0:strm] commitlog - Issuing shutdown for all segments Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 0:strm] commitlog - Checking for unused segments (0 active) Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 0:strm] commitlog - Segment HintsLog-3-28417993.log is no longer active and will submitted for delete now Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 0:strm] commitlog - Discarding segments [/var/lib/scylla/hints/0/51f9ec26-776f-4f66-96f0-51018af4e9b7/HintsLog-3-28417993.log (33554432) (Force Delete)] Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 0:strm] commitlog - Issuing sync for all segments Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 0:strm] commitlog - Checking for unused segments (0 active) Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 0:strm] commitlog - Issuing shutdown for all segments Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 0:strm] commitlog - Checking for unused segments (0 active) Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 0:strm] commitlog - Segment HintsLog-3-28418003.log is no longer active and will submitted for delete now Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 0:strm] commitlog - Discarding segments [/var/lib/scylla/hints/0/81ec2a79-17b5-451e-b6d1-960aab842f3f/HintsLog-3-28418003.log (33554432) (Force Delete)] Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 0:strm] commitlog - Issuing sync for all segments Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 0:strm] commitlog - Checking for unused segments (0 active) Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 0:strm] commitlog - Issuing shutdown for all segments Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 0:strm] commitlog - Checking for unused segments (0 active) Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 0:strm] commitlog - Segment HintsLog-3-28417994.log is no longer active and will submitted for delete now Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 0:strm] commitlog - Discarding segments [/var/lib/scylla/hints/0/dcfb91cf-5d0a-436b-b71a-a554c1bd1bc5/HintsLog-3-28417994.log (33554432) (Force Delete)] Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 0:strm] commitlog - Deleting segment file /var/lib/scylla/hints/0/17b6fc29-812b-4812-bc3a-ecfa256828f2/HintsLog-3-28418000.log Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 0:strm] commitlog - Deleting segment file /var/lib/scylla/hints/0/51f9ec26-776f-4f66-96f0-51018af4e9b7/HintsLog-3-28417993.log Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 0:strm] commitlog - Deleting segment file /var/lib/scylla/hints/0/81ec2a79-17b5-451e-b6d1-960aab842f3f/HintsLog-3-28418003.log Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 0:strm] commitlog - Deleting segment file /var/lib/scylla/hints/0/dcfb91cf-5d0a-436b-b71a-a554c1bd1bc5/HintsLog-3-28417994.log Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 0:strm] commitlog - Commitlog shutdown complete Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 0:strm] commitlog - Commitlog /var/lib/scylla/hints/0/17b6fc29-812b-4812-bc3a-ecfa256828f2 disposed Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 0:strm] commitlog - Commitlog shutdown complete Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 0:strm] commitlog - Commitlog /var/lib/scylla/hints/0/51f9ec26-776f-4f66-96f0-51018af4e9b7 disposed Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 0:strm] commitlog - Commitlog shutdown complete Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 0:strm] commitlog - Commitlog /var/lib/scylla/hints/0/81ec2a79-17b5-451e-b6d1-960aab842f3f disposed Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 0:strm] commitlog - Commitlog shutdown complete Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 0:strm] commitlog - Commitlog /var/lib/scylla/hints/0/dcfb91cf-5d0a-436b-b71a-a554c1bd1bc5 disposed Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 0:strm] commitlog - Commitlog /var/lib/scylla/hints/0/dcfb91cf-5d0a-436b-b71a-a554c1bd1bc5 maximum disk size: 22 MB / cpu (6 cpus) Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 0:strm] commitlog - Commitlog /var/lib/scylla/hints/0/51f9ec26-776f-4f66-96f0-51018af4e9b7 maximum disk size: 22 MB / cpu (6 cpus) Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 0:strm] commitlog - Commitlog /var/lib/scylla/hints/0/81ec2a79-17b5-451e-b6d1-960aab842f3f maximum disk size: 22 MB / cpu (6 cpus) Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 0:strm] commitlog - Commitlog /var/lib/scylla/hints/0/17b6fc29-812b-4812-bc3a-ecfa256828f2 maximum disk size: 22 MB / cpu (6 cpus) Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 0:strm] commitlog - Delaying timer loop 0 ms Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 0:strm] commitlog - Delaying timer loop 0 ms Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 0:strm] commitlog - Delaying timer loop 0 ms Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 0:strm] commitlog - Delaying timer loop 0 ms Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 0:main] commitlog - Created new segment HintsLog-3-28417994.log Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 0:main] commitlog - Created new segment HintsLog-3-28417993.log Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 0:main] commitlog - Created new segment HintsLog-3-28418003.log Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 0:main] commitlog - Created new segment HintsLog-3-28418000.log Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 4:stmt] commitlog - Writing 249 entries, 131072 k in 4468736 -> 4599808 Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 4:stmt] commitlog - Allocated 128 k buffer Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 4:clog] commitlog - Final write of 131072 to CommitLog-3-72057594066346605.log: 131072/131072 bytes at 4468736 Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 4:strm] commitlog - Issuing sync for all segments Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 4:strm] commitlog - Checking for unused segments (0 active) Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 4:strm] commitlog - Issuing shutdown for all segments Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 4:strm] commitlog - Checking for unused segments (0 active) Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 4:strm] commitlog - Segment HintsLog-3-72057594066345939.log is no longer active and will submitted for delete now Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 4:strm] commitlog - Discarding segments [/var/lib/scylla/hints/4/81ec2a79-17b5-451e-b6d1-960aab842f3f/HintsLog-3-72057594066345939.log (33554432) (Force Delete)] Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 4:strm] commitlog - Issuing sync for all segments Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 4:strm] commitlog - Checking for unused segments (0 active) Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 4:strm] commitlog - Issuing shutdown for all segments Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 4:strm] commitlog - Checking for unused segments (0 active) Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 4:strm] commitlog - Segment HintsLog-3-72057594066345937.log is no longer active and will submitted for delete now Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 4:strm] commitlog - Discarding segments [/var/lib/scylla/hints/4/864ba0e1-5a86-4ac9-a1c2-0f75836c13f3/HintsLog-3-72057594066345937.log (33554432) (Force Delete)] Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 4:strm] commitlog - Issuing sync for all segments Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 4:strm] commitlog - Checking for unused segments (0 active) Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 4:strm] commitlog - Issuing shutdown for all segments Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 4:strm] commitlog - Checking for unused segments (0 active) Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 4:strm] commitlog - Segment HintsLog-3-72057594066345930.log is no longer active and will submitted for delete now Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 4:strm] commitlog - Discarding segments [/var/lib/scylla/hints/4/dcfb91cf-5d0a-436b-b71a-a554c1bd1bc5/HintsLog-3-72057594066345930.log (33554432) (Force Delete)] Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 4:strm] commitlog - Issuing sync for all segments Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 4:strm] commitlog - Checking for unused segments (0 active) Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 4:strm] commitlog - Issuing shutdown for all segments Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 4:strm] commitlog - Checking for unused segments (0 active) Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 4:strm] commitlog - Segment HintsLog-3-72057594066345929.log is no longer active and will submitted for delete now Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 4:strm] commitlog - Discarding segments [/var/lib/scylla/hints/4/51f9ec26-776f-4f66-96f0-51018af4e9b7/HintsLog-3-72057594066345929.log (33554432) (Force Delete)] Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 4:strm] commitlog - Deleting segment file /var/lib/scylla/hints/4/81ec2a79-17b5-451e-b6d1-960aab842f3f/HintsLog-3-72057594066345939.log Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 4:strm] commitlog - Deleting segment file /var/lib/scylla/hints/4/864ba0e1-5a86-4ac9-a1c2-0f75836c13f3/HintsLog-3-72057594066345937.log Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 4:strm] commitlog - Deleting segment file /var/lib/scylla/hints/4/dcfb91cf-5d0a-436b-b71a-a554c1bd1bc5/HintsLog-3-72057594066345930.log Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 4:strm] commitlog - Deleting segment file /var/lib/scylla/hints/4/51f9ec26-776f-4f66-96f0-51018af4e9b7/HintsLog-3-72057594066345929.log Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 4:strm] commitlog - Commitlog shutdown complete Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 4:strm] commitlog - Commitlog /var/lib/scylla/hints/4/81ec2a79-17b5-451e-b6d1-960aab842f3f disposed Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 4:strm] commitlog - Commitlog shutdown complete Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 4:strm] commitlog - Commitlog /var/lib/scylla/hints/4/864ba0e1-5a86-4ac9-a1c2-0f75836c13f3 disposed Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 4:strm] commitlog - Commitlog shutdown complete Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 4:strm] commitlog - Commitlog /var/lib/scylla/hints/4/dcfb91cf-5d0a-436b-b71a-a554c1bd1bc5 disposed Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 4:strm] commitlog - Commitlog shutdown complete Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 4:strm] commitlog - Commitlog /var/lib/scylla/hints/4/51f9ec26-776f-4f66-96f0-51018af4e9b7 disposed Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 4:strm] commitlog - Commitlog /var/lib/scylla/hints/4/81ec2a79-17b5-451e-b6d1-960aab842f3f maximum disk size: 22 MB / cpu (6 cpus) Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 4:strm] commitlog - Commitlog /var/lib/scylla/hints/4/dcfb91cf-5d0a-436b-b71a-a554c1bd1bc5 maximum disk size: 22 MB / cpu (6 cpus) Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 4:strm] commitlog - Commitlog /var/lib/scylla/hints/4/51f9ec26-776f-4f66-96f0-51018af4e9b7 maximum disk size: 22 MB / cpu (6 cpus) Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 4:strm] commitlog - Delaying timer loop 6668 ms Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 4:strm] commitlog - Delaying timer loop 6668 ms Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 4:strm] commitlog - Delaying timer loop 6668 ms Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 4:main] commitlog - Created new segment HintsLog-3-72057594066345939.log Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 4:main] commitlog - Created new segment HintsLog-3-72057594066345930.log Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 4:main] commitlog - Created new segment HintsLog-3-72057594066345929.log Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 4:strm] commitlog - Commitlog /var/lib/scylla/hints/4/864ba0e1-5a86-4ac9-a1c2-0f75836c13f3 maximum disk size: 22 MB / cpu (6 cpus) Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 4:strm] commitlog - Delaying timer loop 6668 ms Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 4:main] commitlog - Created new segment HintsLog-3-72057594066345937.log Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 0: gms] commitlog - Allocated 128 k buffer Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 0: gms] commitlog - Writing 1 entries, 4096 k in 122343424 -> 122347520 Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 0: gms] commitlog - Allocated 128 k buffer Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 0:sclg] commitlog - Final write of 4096 to SchemaLog-3-28418171.log: 4096/4096 bytes at 122343424 Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 0: gms] commitlog - SchemaLog-3-28418171.log synced to 122347520 Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 0: gms] commitlog - Writing 5 entries, 4096 k in 122347520 -> 122351616 Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 0:sclg] commitlog - Final write of 4096 to SchemaLog-3-28418171.log: 4096/4096 bytes at 122347520 Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 0: gms] commitlog - SchemaLog-3-28418171.log synced to 122351616 Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 1:strm] commitlog - Issuing sync for all segments Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 1:strm] commitlog - Issuing sync for all segments Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 1:strm] commitlog - Checking for unused segments (0 active) Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 1:strm] commitlog - Issuing shutdown for all segments Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 1:strm] commitlog - Checking for unused segments (0 active) Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 1:strm] commitlog - Segment HintsLog-3-18014398537899987.log is no longer active and will submitted for delete now Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 1:strm] commitlog - Discarding segments [/var/lib/scylla/hints/1/80361f2f-c12f-453e-8d08-01b19dcdf046/HintsLog-3-18014398537899987.log (33554432) (Force Delete)] Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 1:strm] commitlog - Checking for unused segments (0 active) Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 1:strm] commitlog - Issuing shutdown for all segments Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 1:strm] commitlog - Checking for unused segments (0 active) Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 1:strm] commitlog - Segment HintsLog-3-18014398537899988.log is no longer active and will submitted for delete now Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 1:strm] commitlog - Discarding segments [/var/lib/scylla/hints/1/81ec2a79-17b5-451e-b6d1-960aab842f3f/HintsLog-3-18014398537899988.log (33554432) (Force Delete)] Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 1:strm] commitlog - Issuing sync for all segments Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 1:strm] commitlog - Checking for unused segments (0 active) Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 1:strm] commitlog - Issuing shutdown for all segments Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 1:strm] commitlog - Checking for unused segments (0 active) Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 1:strm] commitlog - Segment HintsLog-3-18014398537899978.log is no longer active and will submitted for delete now Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 1:strm] commitlog - Discarding segments [/var/lib/scylla/hints/1/dcfb91cf-5d0a-436b-b71a-a554c1bd1bc5/HintsLog-3-18014398537899978.log (33554432) (Force Delete)] Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 1:strm] commitlog - Issuing sync for all segments Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 1:strm] commitlog - Checking for unused segments (0 active) Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 1:strm] commitlog - Issuing shutdown for all segments Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 1:strm] commitlog - Checking for unused segments (0 active) Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 1:strm] commitlog - Segment HintsLog-3-18014398537899977.log is no longer active and will submitted for delete now Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 1:strm] commitlog - Discarding segments [/var/lib/scylla/hints/1/51f9ec26-776f-4f66-96f0-51018af4e9b7/HintsLog-3-18014398537899977.log (33554432) (Force Delete)] Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 1:strm] commitlog - Deleting segment file /var/lib/scylla/hints/1/80361f2f-c12f-453e-8d08-01b19dcdf046/HintsLog-3-18014398537899987.log Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 1:strm] commitlog - Deleting segment file /var/lib/scylla/hints/1/81ec2a79-17b5-451e-b6d1-960aab842f3f/HintsLog-3-18014398537899988.log Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 1:strm] commitlog - Deleting segment file /var/lib/scylla/hints/1/dcfb91cf-5d0a-436b-b71a-a554c1bd1bc5/HintsLog-3-18014398537899978.log Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 1:strm] commitlog - Deleting segment file /var/lib/scylla/hints/1/51f9ec26-776f-4f66-96f0-51018af4e9b7/HintsLog-3-18014398537899977.log Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 1:strm] commitlog - Commitlog shutdown complete Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 1:strm] commitlog - Commitlog /var/lib/scylla/hints/1/80361f2f-c12f-453e-8d08-01b19dcdf046 disposed Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 1:strm] commitlog - Commitlog shutdown complete Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 1:strm] commitlog - Commitlog /var/lib/scylla/hints/1/81ec2a79-17b5-451e-b6d1-960aab842f3f disposed Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 1:strm] commitlog - Commitlog shutdown complete Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 1:strm] commitlog - Commitlog /var/lib/scylla/hints/1/dcfb91cf-5d0a-436b-b71a-a554c1bd1bc5 disposed Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 1:strm] commitlog - Commitlog shutdown complete Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 1:strm] commitlog - Commitlog /var/lib/scylla/hints/1/51f9ec26-776f-4f66-96f0-51018af4e9b7 disposed Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 1:strm] commitlog - Commitlog /var/lib/scylla/hints/1/51f9ec26-776f-4f66-96f0-51018af4e9b7 maximum disk size: 22 MB / cpu (6 cpus) Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 1:strm] commitlog - Commitlog /var/lib/scylla/hints/1/80361f2f-c12f-453e-8d08-01b19dcdf046 maximum disk size: 22 MB / cpu (6 cpus) Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 1:strm] commitlog - Commitlog /var/lib/scylla/hints/1/81ec2a79-17b5-451e-b6d1-960aab842f3f maximum disk size: 22 MB / cpu (6 cpus) Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 1:strm] commitlog - Delaying timer loop 1667 ms Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 1:strm] commitlog - Delaying timer loop 1667 ms Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 1:strm] commitlog - Delaying timer loop 1667 ms Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 1:main] commitlog - Created new segment HintsLog-3-18014398537899977.log Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 1:strm] commitlog - Commitlog /var/lib/scylla/hints/1/dcfb91cf-5d0a-436b-b71a-a554c1bd1bc5 maximum disk size: 22 MB / cpu (6 cpus) Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 1:main] commitlog - Created new segment HintsLog-3-18014398537899987.log Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 1:main] commitlog - Created new segment HintsLog-3-18014398537899988.log Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 1:strm] commitlog - Delaying timer loop 1667 ms Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 1:main] commitlog - Created new segment HintsLog-3-18014398537899978.log Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 2:strm] commitlog - Issuing sync for all segments Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 2:strm] commitlog - Checking for unused segments (0 active) Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 2:strm] commitlog - Issuing shutdown for all segments Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 2:strm] commitlog - Checking for unused segments (0 active) Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 2:strm] commitlog - Segment HintsLog-3-36028797047381968.log is no longer active and will submitted for delete now Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 2:strm] commitlog - Discarding segments [/var/lib/scylla/hints/2/17b6fc29-812b-4812-bc3a-ecfa256828f2/HintsLog-3-36028797047381968.log (33554432) (Force Delete)] Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 2:strm] commitlog - Issuing sync for all segments Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 2:strm] commitlog - Checking for unused segments (0 active) Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 2:strm] commitlog - Issuing shutdown for all segments Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 2:strm] commitlog - Checking for unused segments (0 active) Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 2:strm] commitlog - Segment HintsLog-3-36028797047381961.log is no longer active and will submitted for delete now Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 2:strm] commitlog - Discarding segments [/var/lib/scylla/hints/2/51f9ec26-776f-4f66-96f0-51018af4e9b7/HintsLog-3-36028797047381961.log (33554432) (Force Delete)] Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 2:strm] commitlog - Issuing sync for all segments Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 2:strm] commitlog - Checking for unused segments (0 active) Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 2:strm] commitlog - Issuing shutdown for all segments Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 2:strm] commitlog - Checking for unused segments (0 active) Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 2:strm] commitlog - Segment HintsLog-3-36028797047381961.log is no longer active and will submitted for delete now Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 2:strm] commitlog - Discarding segments [/var/lib/scylla/hints/2/dcfb91cf-5d0a-436b-b71a-a554c1bd1bc5/HintsLog-3-36028797047381961.log (33554432) (Force Delete)] Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 2:strm] commitlog - Issuing sync for all segments Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 2:strm] commitlog - Checking for unused segments (0 active) Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 2:strm] commitlog - Issuing shutdown for all segments Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 2:strm] commitlog - Checking for unused segments (0 active) Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 2:strm] commitlog - Segment HintsLog-3-36028797047381971.log is no longer active and will submitted for delete now Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 2:strm] commitlog - Discarding segments [/var/lib/scylla/hints/2/81ec2a79-17b5-451e-b6d1-960aab842f3f/HintsLog-3-36028797047381971.log (33554432) (Force Delete)] Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 2:strm] commitlog - Deleting segment file /var/lib/scylla/hints/2/17b6fc29-812b-4812-bc3a-ecfa256828f2/HintsLog-3-36028797047381968.log Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 2:strm] commitlog - Deleting segment file /var/lib/scylla/hints/2/51f9ec26-776f-4f66-96f0-51018af4e9b7/HintsLog-3-36028797047381961.log Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 2:strm] commitlog - Deleting segment file /var/lib/scylla/hints/2/dcfb91cf-5d0a-436b-b71a-a554c1bd1bc5/HintsLog-3-36028797047381961.log Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 2:strm] commitlog - Deleting segment file /var/lib/scylla/hints/2/81ec2a79-17b5-451e-b6d1-960aab842f3f/HintsLog-3-36028797047381971.log Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 2:strm] commitlog - Commitlog shutdown complete Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 2:strm] commitlog - Commitlog /var/lib/scylla/hints/2/17b6fc29-812b-4812-bc3a-ecfa256828f2 disposed Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 2:strm] commitlog - Commitlog shutdown complete Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 2:strm] commitlog - Commitlog /var/lib/scylla/hints/2/51f9ec26-776f-4f66-96f0-51018af4e9b7 disposed Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 2:strm] commitlog - Commitlog shutdown complete Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 2:strm] commitlog - Commitlog /var/lib/scylla/hints/2/dcfb91cf-5d0a-436b-b71a-a554c1bd1bc5 disposed Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 2:strm] commitlog - Commitlog shutdown complete Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 2:strm] commitlog - Commitlog /var/lib/scylla/hints/2/81ec2a79-17b5-451e-b6d1-960aab842f3f disposed Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 2:strm] commitlog - Issuing sync for all segments Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 2:strm] commitlog - Checking for unused segments (0 active) Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 2:strm] commitlog - Issuing shutdown for all segments Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 2:strm] commitlog - Checking for unused segments (0 active) Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 2:strm] commitlog - Segment HintsLog-3-36028797047381971.log is no longer active and will submitted for delete now Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 2:strm] commitlog - Discarding segments [/var/lib/scylla/hints/2/80361f2f-c12f-453e-8d08-01b19dcdf046/HintsLog-3-36028797047381971.log (33554432) (Force Delete)] Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 2:strm] commitlog - Deleting segment file /var/lib/scylla/hints/2/80361f2f-c12f-453e-8d08-01b19dcdf046/HintsLog-3-36028797047381971.log Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 2:strm] commitlog - Commitlog shutdown complete Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 2:strm] commitlog - Commitlog /var/lib/scylla/hints/2/80361f2f-c12f-453e-8d08-01b19dcdf046 disposed Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 2:strm] commitlog - Commitlog /var/lib/scylla/hints/2/81ec2a79-17b5-451e-b6d1-960aab842f3f maximum disk size: 22 MB / cpu (6 cpus) Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 2:strm] commitlog - Commitlog /var/lib/scylla/hints/2/17b6fc29-812b-4812-bc3a-ecfa256828f2 maximum disk size: 22 MB / cpu (6 cpus) Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 2:strm] commitlog - Commitlog /var/lib/scylla/hints/2/51f9ec26-776f-4f66-96f0-51018af4e9b7 maximum disk size: 22 MB / cpu (6 cpus) Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 2:strm] commitlog - Delaying timer loop 3334 ms Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 2:strm] commitlog - Delaying timer loop 3334 ms Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 2:strm] commitlog - Delaying timer loop 3334 ms Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 2:main] commitlog - Created new segment HintsLog-3-36028797047381971.log Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 2:main] commitlog - Created new segment HintsLog-3-36028797047381968.log Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 2:strm] commitlog - Commitlog /var/lib/scylla/hints/2/dcfb91cf-5d0a-436b-b71a-a554c1bd1bc5 maximum disk size: 22 MB / cpu (6 cpus) Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 2:strm] commitlog - Commitlog /var/lib/scylla/hints/2/80361f2f-c12f-453e-8d08-01b19dcdf046 maximum disk size: 22 MB / cpu (6 cpus) Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 2:main] commitlog - Created new segment HintsLog-3-36028797047381961.log Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 2:strm] commitlog - Delaying timer loop 3334 ms Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 2:strm] commitlog - Delaying timer loop 3334 ms Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 2:main] commitlog - Created new segment HintsLog-3-36028797047381961.log Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 2:main] commitlog - Created new segment HintsLog-3-36028797047381971.log Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 5:strm] commitlog - Issuing sync for all segments Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 5:strm] commitlog - Checking for unused segments (0 active) Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 5:strm] commitlog - Issuing shutdown for all segments Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 5:strm] commitlog - Checking for unused segments (0 active) Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 5:strm] commitlog - Segment HintsLog-3-90071992575827924.log is no longer active and will submitted for delete now Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 5:strm] commitlog - Discarding segments [/var/lib/scylla/hints/5/80361f2f-c12f-453e-8d08-01b19dcdf046/HintsLog-3-90071992575827924.log (33554432) (Force Delete)] Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 5:strm] commitlog - Issuing sync for all segments Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 5:strm] commitlog - Checking for unused segments (0 active) Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 5:strm] commitlog - Issuing shutdown for all segments Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 5:strm] commitlog - Checking for unused segments (0 active) Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 5:strm] commitlog - Segment HintsLog-3-90071992575827923.log is no longer active and will submitted for delete now Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 5:strm] commitlog - Discarding segments [/var/lib/scylla/hints/5/81ec2a79-17b5-451e-b6d1-960aab842f3f/HintsLog-3-90071992575827923.log (33554432) (Force Delete)] Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 5:strm] commitlog - Issuing sync for all segments Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 5:strm] commitlog - Checking for unused segments (0 active) Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 5:strm] commitlog - Issuing shutdown for all segments Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 5:strm] commitlog - Checking for unused segments (0 active) Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 5:strm] commitlog - Segment HintsLog-3-90071992575827920.log is no longer active and will submitted for delete now Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 5:strm] commitlog - Discarding segments [/var/lib/scylla/hints/5/17b6fc29-812b-4812-bc3a-ecfa256828f2/HintsLog-3-90071992575827920.log (33554432) (Force Delete)] Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 5:strm] commitlog - Issuing sync for all segments Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 5:strm] commitlog - Checking for unused segments (0 active) Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 5:strm] commitlog - Issuing shutdown for all segments Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 5:strm] commitlog - Checking for unused segments (0 active) Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 5:strm] commitlog - Segment HintsLog-3-90071992575827913.log is no longer active and will submitted for delete now Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 5:strm] commitlog - Discarding segments [/var/lib/scylla/hints/5/51f9ec26-776f-4f66-96f0-51018af4e9b7/HintsLog-3-90071992575827913.log (33554432) (Force Delete)] Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 5:strm] commitlog - Issuing sync for all segments Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 5:strm] commitlog - Checking for unused segments (0 active) Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 5:strm] commitlog - Issuing shutdown for all segments Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 5:strm] commitlog - Checking for unused segments (0 active) Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 5:strm] commitlog - Segment HintsLog-3-90071992575827914.log is no longer active and will submitted for delete now Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 5:strm] commitlog - Discarding segments [/var/lib/scylla/hints/5/dcfb91cf-5d0a-436b-b71a-a554c1bd1bc5/HintsLog-3-90071992575827914.log (33554432) (Force Delete)] Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 5:strm] commitlog - Deleting segment file /var/lib/scylla/hints/5/80361f2f-c12f-453e-8d08-01b19dcdf046/HintsLog-3-90071992575827924.log Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 5:strm] commitlog - Deleting segment file /var/lib/scylla/hints/5/81ec2a79-17b5-451e-b6d1-960aab842f3f/HintsLog-3-90071992575827923.log Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 5:strm] commitlog - Deleting segment file /var/lib/scylla/hints/5/17b6fc29-812b-4812-bc3a-ecfa256828f2/HintsLog-3-90071992575827920.log Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 5:strm] commitlog - Deleting segment file /var/lib/scylla/hints/5/51f9ec26-776f-4f66-96f0-51018af4e9b7/HintsLog-3-90071992575827913.log Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 5:strm] commitlog - Deleting segment file /var/lib/scylla/hints/5/dcfb91cf-5d0a-436b-b71a-a554c1bd1bc5/HintsLog-3-90071992575827914.log Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 5:strm] commitlog - Commitlog shutdown complete Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 5:strm] commitlog - Commitlog /var/lib/scylla/hints/5/80361f2f-c12f-453e-8d08-01b19dcdf046 disposed Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 5:strm] commitlog - Commitlog shutdown complete Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 5:strm] commitlog - Commitlog /var/lib/scylla/hints/5/81ec2a79-17b5-451e-b6d1-960aab842f3f disposed Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 5:strm] commitlog - Commitlog shutdown complete Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 5:strm] commitlog - Commitlog /var/lib/scylla/hints/5/17b6fc29-812b-4812-bc3a-ecfa256828f2 disposed Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 5:strm] commitlog - Commitlog shutdown complete Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 5:strm] commitlog - Commitlog /var/lib/scylla/hints/5/51f9ec26-776f-4f66-96f0-51018af4e9b7 disposed Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 5:strm] commitlog - Commitlog shutdown complete Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 5:strm] commitlog - Commitlog /var/lib/scylla/hints/5/dcfb91cf-5d0a-436b-b71a-a554c1bd1bc5 disposed Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 5:strm] commitlog - Commitlog /var/lib/scylla/hints/5/51f9ec26-776f-4f66-96f0-51018af4e9b7 maximum disk size: 22 MB / cpu (6 cpus) Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 5:strm] commitlog - Commitlog /var/lib/scylla/hints/5/81ec2a79-17b5-451e-b6d1-960aab842f3f maximum disk size: 22 MB / cpu (6 cpus) Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 5:strm] commitlog - Commitlog /var/lib/scylla/hints/5/dcfb91cf-5d0a-436b-b71a-a554c1bd1bc5 maximum disk size: 22 MB / cpu (6 cpus) Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 5:strm] commitlog - Commitlog /var/lib/scylla/hints/5/80361f2f-c12f-453e-8d08-01b19dcdf046 maximum disk size: 22 MB / cpu (6 cpus) Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 5:strm] commitlog - Commitlog /var/lib/scylla/hints/5/17b6fc29-812b-4812-bc3a-ecfa256828f2 maximum disk size: 22 MB / cpu (6 cpus) Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 5:strm] commitlog - Delaying timer loop 8335 ms Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 5:strm] commitlog - Delaying timer loop 8335 ms Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 5:strm] commitlog - Delaying timer loop 8335 ms Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 5:strm] commitlog - Delaying timer loop 8335 ms Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 5:strm] commitlog - Delaying timer loop 8335 ms Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 5:main] commitlog - Created new segment HintsLog-3-90071992575827913.log Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 5:main] commitlog - Created new segment HintsLog-3-90071992575827923.log Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 5:main] commitlog - Created new segment HintsLog-3-90071992575827914.log Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 5:main] commitlog - Created new segment HintsLog-3-90071992575827924.log Oct 07 12:02:50 o-p-L3-8 scylla[32999]: [shard 5:main] commitlog - Created new segment HintsLog-3-90071992575827920.log ``` A (second) restart of the upgraded or new node fixes the problem. So whenever we upgrade a cluster to 6.0 or we add new nodes to a 6.0 cluster, it seems we have to do a second scylla restart to fix this.