Databricks Community

Hi @Alberto_Umana thank you for your response. Unfortunately we already have all these three things in place. The problem still remains as when a member of the group alters a view, they might become the new owner, which can restrict other members fro...

@stevenayers-bge Autoloader is designed to work best with immutable files. If files are mutable (i.e., they can be updated), it is recommended to set cloudFiles.allowOverwrites = true to ensure that the latest version of the file is read. Please refe...

@Marcin_U Please use the below option in the readStream to load only parquet files .option("pathGlobfilter", "*.parquet") Please refer to the below documentation: https://docs.databricks.com/en/ingestion/cloud-object-storage/auto-loader/options.html...

 Hello,Thanks for your quick update. I will continue with the setup. I also have one more question. It is mentioned in the set up that, we have to have the bellow in the Kafka client properties.In addition, if you choose to configure your connection ...

Hi @sashikanth, I would recommend opening a case with us further investigate this behavior since this requires a code logic review and additional checks. Please refer to: https://docs.databricks.com/en/resources/support.html

@Youngwb Did you fixed this issue?

Hi,I can be accessed with the following code.storageAccountName = "my-storage-account-name" applicationClientId = "my-umi-client-id" aadDirectoryId = "my-entra-tenant-id" containerName = "my-lake-container" spark.conf.set("fs.azure.account.auth.type...

Hi,Could you please share the error message or any logs you’re seeing when the task gets stuck? This will help in diagnosing the issue and identifying potential solutions.

Thanks for sharing. In the original screenshots I've noticed that you've set delta.isolationLevel to Serializable, which is the strongest (and most strict) level. Please try WriteSerializable, which is the default level.

If you intend to capture data changes, take a look at this doc, which talks about change data feed in Databricks.

Hey @nalindhar I assume you want the target table to have the same name once you declare your ETL operations in the DLT pipeline, if that's the case, begin with renaming your delta table to specify that they contain historical data, you can do this v...

Hi @kfloresip ,As per documentation you need to run:%pip install --index-url=https://blpapi.bloomberg.com/repository/releases/python/simple/ blpapi It means that the package is not available on the standard Python Package Index (PyPI) and Bloomberg h...