Originally from the User Slack
@Nishant_Kumar: Hii, I have a table, in which for few column type is UDT, for some reason I can’t use UDT and want to change it to text. Is it supported in scylla?
I tried with alter type query locally Scylla 6.0.1, it didn’t work.
Any way to do it? other than creating a new table and migrating the data?
Your help highly appreciated.
@Botond_Dénes: In ScyllaDB you can only alter the type of columns to compatible types. So e.g. you can change from int to bigint, but not the other way around.
There are no compatible types that you can convert a UDT to, so the only option is to migrate to a new table.
What is the reason why you can’t use UDT? Is it some ScyllaDB problem, or is it something on the app/business side?
@Nishant_Kumar: @Botond_Dénes We need to put scylla db data to athena for analytics purposes, currently it’s executed BI_HOURLY, and at this time we are seeing spike in latency.
bcz data-platform team is saying
Scylla Debezium CDC doesn’t have support for collection types (LIST, SET, MAP) and UDT - columns. These columns are omitted before Debezium CDC processing.
For handling complex data types, we currently scan the entire CDC table, which retains two days of data, and subsequently process it to extract newly updated or inserted records. This full table scan is necessary because Scylla CDC tables store timestamps as UUIDs, which cannot directly track newer records. As a result, we bring in all the data, and the UUIDs are then converted into DateTime format for further processing.
is there any way to solve this?
For ref:
https://opensource.docs.scylladb.com/stable/using-scylla/integrations/scylla-cdc-source-connector.html
https://opensource.docs.scylladb.com/stable/features/cdc/cdc-log-table.html#time-column
ScyllaDB CDC Source Connector | ScyllaDB Docs
The CDC Log Table | ScyllaDB Docs
@Md_Anees: @Botond_Dénes https://github.com/scylladb/scylla-cdc-source-connector/issues/9
GitHub: feature request: support for collection types (LIST, SET, MAP) and UDT · Issue #9 · scylladb/scylla-cdc-source-connector
We are heavily using UDT (frozen) in our tables, which is causing issues while adding them to CDC table. Any alternates would help.
@Botond_Dénes: Sorry, I’m not aware of a workaround for this.
@Karol_Baryła: https://github.com/scylladb/scylla-cdc-source-connector/pull/21 adds collections support to scylla-cdc-connector. I’ve raised the topic of merging it and releasing a new version recently - cc @Wojciech_Bączkowski @Dmitry_Kropachev
GitHub: Support for collections by Lorak-mmk · Pull Request #21 · scylladb/scylla-cdc-source-connector