Databricks Community

dawidch

Hey.We have different number of topics per let's call it "subject". The connector works great for us if there are not so many topics (partitions) in subject. We have issue when we try to sink 1500 topics, 3 partitions each. We've sharded the topics i...

dawidch

Hello All.Thank you all for help. Here is what worked for us:## Original symptomsEach connector started normally and consumed records for more than an hour. Kafka Connect reportedthe connector and tasks as `RUNNING`, and source-topic lag decreased. T...

dawidch

Thanks for replay!I've tried with ootb 11.1.0 connector with the same bundled iceberg. I've seen that github pull request, I've downloaded the code, built it and run, but I've got the same issue. It seems like the fix doesn't work for me or I did sth...

dawidch

Thanks for quick replay. Is there an open ticket for this somewhere?

Databricks Community

User Stats

User Activity

org.apache.iceberg.connect.IcebergSinkConnector to sink data from kafka to databricks - large volume

Re: org.apache.iceberg.connect.IcebergSinkConnector to sink data from kafka to databricks - large vo

Re: org.apache.iceberg.connect.IcebergSinkConnector to sink data from kafka to databricks - large vo

Re: org.apache.iceberg.connect.IcebergSinkConnector to sink data from kafka to databricks - large vo