Will inconsistencies between the base table and the GSI table be detected?

ScyllaDB
, ,
1

I tried deleting some SSTables files of the base table, then restarting scylla and performing repair on only the base table. At this time, I found that new data will be generated in the memtable, and the data is written to the GSI table. When the base table data changes, will a copy of the data be sent to the GSI synchronously?

This behavior looks identical to the call chain for put_item.

2

database::do_apply(schema_ptr s, → push_view_replica_updates - > do_push_view_replica_updates(schema_ptr s, mutation m, → generate_and_propagate_view_updates(const schema_ptr& base

How does nodetool repair relate to database::do_apply?

3

Repair does not use the regular write path, so data written by repair will not go through database::apply().
Repair writes data to sstables directly. If the repaired table has views or indexes attached to it, the sstable will be registered with the view builder.
See make_streaming_consumer() in streaming/consumer.cc, in particular the call to register_staging_sstable(). Sstables which are registered with the view builder, will be processed and the view builder will generate view updates for all the writes contained in them.

1 Like
4

get_missing_rows_from_follower_nodes → flush_rows_in_working_row_buf → flush_rows → repair_writer_impl::create_writer → streaming::make_streaming_consumer → view_update_generator::register_staging_sstable → ??? → database::do_apply(schema_ptr s,

@Botond_Denes
How is register_staging_sstable propagated to database::do_apply? It is known that when I perform repair on the base table, it will inevitably use the database::do_apply function, as shown in the figure above.

Thanks!

5

View updates generated by the view_builder, will be sent to the relevant view replicas, where they will go through database::apply().

But the repaired data of the repaired table (base table) will not go through database::apply().

6

There is a problem of non-key attributes being lost when the repair base table propagates data to GSI, see Major bug: Repair will cause index table data loss!. Can you help me take a look? The entire call chain logic is quite complicated, and I haven’t understood this code yet.

7

The call chain is as follows:

  • repair/streaming calls view_update_generator::register_streaming_sstable()
  • registered sstable’s content is eventually consumed by db::view::view_updating_consumer.
  • db::view::view_updating_consumer forwards individual rows to replica::table::stream_view_replica_updates()
  • → replica::table::do_push_view_replica_updates()
  • → db::view::view_update_generator::generate_and_propagate_view_updates()
  • → db::view::view_update_generator::mutate_MV()

As you can see, the code-path joins-in with that of the normal write path. They all end up calling mutate_MV() in the end.

1 Like