# Debezium query CDC streams

**URL:** <https://forum.scylladb.com/t/debezium-query-cdc-streams/2308>\
**Category:** ScyllaDB\
**Tags:** ttl, cdc, kafka\
**Created:** [July 15, 2024, 4:05pm UTC](https://forum.scylladb.com/t/debezium-query-cdc-streams/2308 "2024-07-15T16:05:10Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![amitesh88](https://sea2.discourse-cdn.com/flex016/user_avatar/forum.scylladb.com/amitesh88/32/638_2.png) [@amitesh88](https://forum.scylladb.com/u/amitesh88)\
**Post date:** [July 15, 2024, 4:05pm UTC](https://forum.scylladb.com/t/debezium-query-cdc-streams/2308/1 "2024-07-15T16:05:10Z")

</div>

Hi I am using debezium Kafka connect to read CDC log table of ScyllaDB . I am willing to understand ,  
how does debezium query CDC streams for the FIRST TIME ,somewhere i have read it queries now() - ttl of cdc table  
, how does it maintains and use timestamp in connect-offset topic ?  
how it uses offset to polls queries after scylla.query.time.window.size ?

If there is any link or tutorial on this , it would be helpful to understand

---

<div class="post-metadata">

**Author:** ![Bouncheck](https://avatars.discourse-cdn.com/v4/letter/b/b5a626/32.png) [@Bouncheck](https://forum.scylladb.com/u/Bouncheck)\
**Post date:** [December 16, 2024, 1:03pm UTC](https://forum.scylladb.com/t/debezium-query-cdc-streams/2308/2 "2024-12-16T13:03:53Z")

</div>

Hi,  
Currently available resources I know of right now are the following:  
[The readme of CDC connector](https://github.com/scylladb/scylla-cdc-source-connector/blob/master/README.md),  
[readme of scylla-cdc-java](https://github.com/scylladb/scylla-cdc-java/blob/master/README.md) (which is used underneath),  
[scylla-cdc-java printer readme](https://github.com/scylladb/scylla-cdc-java/blob/master/scylla-cdc-printer/README.md) (recommended read; points to replicator example as a follow up).  
Additionally there is [Scylla CDC documentation](https://opensource.docs.scylladb.com/stable/features/cdc/index.html)  
and [“ScyllaDB university” resources](https://university.scylladb.com/courses/data-modeling/lessons/change-data-capture-cdc/topic/change-data-capture-cdc-overview/) about CDC

In regards to specifics of connector your best bet is probably looking at the source code directly. Many of the [configuration options](https://github.com/scylladb/scylla-cdc-source-connector/blob/master/src/main/java/com/scylladb/cdc/debezium/connector/ScyllaConnectorConfig.java) have wordy descriptions, but if the Kafka platform you’re using does not provide GUI, you may haven’t had the opportunity to see them.

> how does debezium query CDC streams for the FIRST TIME ,somewhere i have read it queries now() - ttl of cdc table

That would be correct. There is no use to query earlier data anyway - there shouldn’t be any.  
I believe this is relevant section from scylla-cdc-java - `Worker.java#createTasksWithState()`

```auto
        // The furthest point in time where there might be
        // a CDC change, given table's TTL.
        Map<TableName, Timestamp> minimumWindowStarts = new HashMap<>();

        for (TableName tableName : tableNames) {
            Optional<Long> ttl = workerConfiguration.cql.fetchTableTTL(tableName).get();
            Date minimumWindowStart = new Date(0);
            if (ttl.isPresent()) {
                minimumWindowStart = new Date(now.getTime() - 1000L * ttl.get()); // TTL is in seconds, getTime() in milliseconds
            }
            minimumWindowStarts.put(tableName, new Timestamp(minimumWindowStart));
        }

```

For more insight on how does the cdc connector maintains offsets see the [TaskStateOffsetContext class](https://github.com/scylladb/scylla-cdc-source-connector/blob/master/src/main/java/com/scylladb/cdc/debezium/connector/TaskStateOffsetContext.java). I believe this is what holds the relevant information about the current progress. If you want to trace how a singular row is processed by connector see consume method of [ScyllaChangesConsumer](https://github.com/scylladb/scylla-cdc-source-connector/blob/master/src/main/java/com/scylladb/cdc/debezium/connector/ScyllaChangesConsumer.java#L41).

> how it uses offset to polls queries after scylla.query.time.window.size ?

After processing current window connector should proceed to the next window of the same size.