topic Re: Delta Live Tables with EventHub in Data Engineering

Delta Live Tables with EventHub

repcak — Wed, 25 May 2022 10:31:41 GMT

Hello,

I would like to integrate Databricks Delta Live Tables with Eventhub, but i cannot install com.microsoft.azure:azure-eventhubs-spark_2.12:2.3.17 on delta live cluster.

I tried installed in using Init script (by adding it in Json cluster settings but currently init scripts are not supported

"init_scripts": [

{

"dbfs": {

"destination": "dbfs:/FileStore/Init_scripts/install_eventhub_library.sh"

}

]

Is it possible to install jars using pip?

I also tried install it by dbutils.library.install(dbfs_path) but ity's also only for python packages.

Are there other ways to install the jar/maven library on DLT cluster?

Re: Delta Live Tables with EventHub

repcak — Thu, 26 May 2022 08:48:29 GMT

Ok, i got an answer on stackoverflow 🙂 https://stackoverflow.com/questions/72360893/delta-live-tables-with-eventhub

Re: Delta Live Tables with EventHub

Atanu — Thu, 26 May 2022 10:00:37 GMT

I think this has some details https://docs.microsoft.com/en-us/azure/event-hubs/event-hubs-kafka-spark-tutorial @Kacper Mucha is the issue resolved ?

Re: Delta Live Tables with EventHub

repcak — Thu, 26 May 2022 13:11:29 GMT

Done 🙂

Re: Delta Live Tables with EventHub

repcak — Thu, 26 May 2022 13:11:46 GMT

Yes, issue has been resolved, Thank you