Databricks Community

talenik · 08-30-2024

Hi Everyone, I am trying to access DBFS files while cluster is starting in init script on GCP databricks, but I am not able to list files which are there on DBFS. I tried to download files from GCS bucket as well but init script throws timeout errors...

talenik · 06-27-2024

Hi everyone, I wanted to ask if there is any way through which we can ingest logs from GCP databricks to azure log analytics in store-sync fashion. Meaning we will save logs into some cloud bucket lets say, then from there we should be able to send l...

talenik · 07-30-2024

Hi @Retired_mod ,Thanks for help. We decided to develop our own library for logging to azure log analytics. We used buffer for this. We are currently on timer based logs but in future versions we wanted to move to memory based.Thanks,Nikhil

talenik · 07-30-2024

Hi @Retired_mod , I have few queries on Directory Names with Column Mapping. I have this delta table on ADLS and I am trying to read it, but I am getting below error. How can we read delta tables with column mapping enabled with pyspark?Can you pleas...

talenik · 06-28-2024

Hello @Retired_mod ,Thanks for your help. Sending logs at real-time with workspace URL sounds good but it can add delay in your jobs execution since requests will be triggered sequentially.We are looking something like spark-monitoring for GCP databr...

Databricks Community

User Stats

User Activity

Not able to access dbfs in init script GCP databricks

Ingesting logs from Databricks (GCP) to Azure log Analytics

Re: Ingesting logs from Databricks (GCP) to Azure log Analytics

Re: Delta table partition directories when column mapping is enabled

Re: Ingesting logs from Databricks (GCP) to Azure log Analytics