Databricks Community

Hi, for a Spark job which does some custom computation, I need to access data from a MongoDB collection and access the elements as of type Document. The reason for this is, that I want to apply some custom type serialization which is already implemen...

Hi, we are managing our Databricks Workspaces entirely using terraform. As of today, the module to read the current clusters fails with a timeout. We use this:data "databricks_clusters" "this" { }This has worked flawlessly before. However, now we are...

Hi, I implemented a job that should incrementally read all the available data from a Kinesis Data Stream and terminate afterwards. I schedule the job daily. The data retention period of the data stream is 7 days, i.e., there should be enough time to ...