Databricks Community

8b1tz · 07-21-2024

Hello, I would like to know the best way to insert Datafactory activity logs into my Databricks delta table, so that I can use dashbosrd and create monitoring in Databricks itself , can you help me? I would like every 5 minutes for all activity logs ...

8b1tz · 07-18-2024

Hi Databricks Community,I am looking for a solution to efficiently integrate Azure Data Factory pipeline logs with Databricks at minimal cost. Currently, I have a dashboard that consumes data from a Delta table, and I would like to augment this table...

8b1tz · 07-25-2024

Just to let you know: it worked! Thank you very much, really.

8b1tz · 07-24-2024

but the autoloader is not streaming? It needs to run continuously, I can't understand how it can run this way, could you explain it to me better?

8b1tz · 07-24-2024

My fear is just that the price will be absurdly expensive, can this happen? In the end, it does not need to be in real time, it can be inserted every 10 minutes for example.

8b1tz · 07-24-2024

yes, in the end I just want to put the logs in the delta table and that's it, I don't want to store anything, I know that maybe sending the logs to storage might not be a good option but I tried the event hub a lot and I couldn't :(. in the end I thi...

8b1tz · 07-23-2024

I have a question: Should I use only the job trigger and a notebook without Auto Loader, use only the Auto Loader, or use the job trigger along with the Auto Loader?

Databricks Community

User Stats

User Activity

ADF logs into Databricks

Data factory logs into databricks delta table

Re: ADF logs into Databricks

Re: ADF logs into Databricks

Re: ADF logs into Databricks

Re: ADF logs into Databricks

Re: ADF logs into Databricks