topic Re: Databricks notebook command logging in Data Engineering

Databricks notebook command logging

C_1 — Fri, 01 Apr 2022 04:06:52 GMT

Hello Community,

I am trying to search for Databricks notebook command logging feature for compliance purpose.

My requirement is to log the exact spark sql fired by user.

I didnt get spark sql (notebook command) tracked under this azure diagnostic logs.

https://docs.microsoft.com/en-us/azure/databricks/administration-guide/account-settings/azure-diagnostic-logs

Any help will be appreciated?

Re: Databricks notebook command logging

C_1 — Mon, 04 Apr 2022 09:12:02 GMT

Hi @Kaniz Fatma ,

yes we are having premium plan.

could you please confirm if spark SQL query fired by user get track under Diagnostic logs or getting track somewhere else?

Re: Databricks notebook command logging

Noopur_Nigam — Fri, 13 May 2022 15:16:55 GMT

Hi @C P we don't have this feature implemented, however, there is already an existing idea available in our idea portal here: https://databricks.aha.io/features/DB-7583.

You can check and vote the same.

Re: Databricks notebook command logging

C_1 — Fri, 13 May 2022 15:35:21 GMT

Hi @Noopur Nigam ,

i am not able to login to above databricks.aha portal.

could you pls allow me to login or paste the idea here?

thanks for replying!!

Re: Databricks notebook command logging

jose_gonzalez — Tue, 07 Jun 2022 18:01:38 GMT

Hi @C P,

Just a friendly follow-up. Did you miss Kaniz's response?

Re: Databricks notebook command logging

jose_gonzalez — Mon, 25 Jul 2022 18:39:47 GMT

HI @C P,

Just a friendly follow-up. Did any of the responses help you to resolve your question? if it did, please mark it as best. Otherwise, please let us know if you still need help.