cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

MrJava
by New Contributor III
  • 3408 Views
  • 9 replies
  • 9 kudos

How to know, who started a job run?

Hi there!We have different jobs/workflows configured in our Databricks workspace running on AWS and would like to know who actually started the job run? Are they started by a user or a service principle using curl?Currently one can only see, who is t...

  • 3408 Views
  • 9 replies
  • 9 kudos
Latest Reply
leonorgrosso
New Contributor
  • 9 kudos

I've just posted this idea on the Idea Portal of Databricks regarding this subject. Upvote it so it may get developed!https://feedback.azure.com/d365community/idea/5d0fdbbf-eefb-ee11-a73c-0022485313bb

  • 9 kudos
8 More Replies
Ajay-Pandey
by Esteemed Contributor III
  • 2638 Views
  • 5 replies
  • 0 kudos

How we can send databricks log to Azure Application Insight ?

Hi All,I want to send databricks logs to azure application insight.Is there any way we can do it ??Any blog or doc will help me.

  • 2638 Views
  • 5 replies
  • 0 kudos
Latest Reply
floringrigoriu
New Contributor II
  • 0 kudos

hi @Debayan in the  https://learn.microsoft.com/en-us/azure/architecture/databricks-monitoring/application-logs. there is a github repository mentioned https://github.com/mspnp/spark-monitoring ? That repository is marked as  maintainance mode.  Just...

  • 0 kudos
4 More Replies
Charmin
by New Contributor
  • 548 Views
  • 1 replies
  • 0 kudos

Why 'runCommand' action does NOT show up in databricksNotebook audit log table?

I understand databricks can send diagnostic/audit logs to log analytics in azure. There is a standard 'DatabricksNotebook' table that provides audit log for notebook actions. In this table there is an action called 'runCommand' but this does not show...

  • 548 Views
  • 1 replies
  • 0 kudos
Latest Reply
rsamant07
New Contributor III
  • 0 kudos

Hi @Charmin patel​  , you need to enable verbose audit logging in workspace setting for runCommand to appear in the audit logs

  • 0 kudos
Mado
by Valued Contributor II
  • 2430 Views
  • 4 replies
  • 3 kudos

Resolved! Databricks Audit Logs, What is "dataSourceId"?

Hi,I want to access the Databricks Audit Logs to check user activity.I created a Databricks workspace on the premium pricing tier.I configured Audit logs to be sent to Azure Diagnostic log delivery. What I got in the "Log Analytics Workspace":  I hav...

image image image
  • 2430 Views
  • 4 replies
  • 3 kudos
Latest Reply
youssefmrini
Honored Contributor III
  • 3 kudos

The data_source_id field specifies the id of the SQL warehouse against which this query will run. You can use the Data Sources API to see a complete list of available SQL warehouses.

  • 3 kudos
3 More Replies
Mado
by Valued Contributor II
  • 2332 Views
  • 3 replies
  • 2 kudos

Databricks Audit Logs, Where the log files are stored? How to read them?

Hi,I want to access the Databricks Audit Logs to check user activity. For example, the number of times that a table was viewed by a user.I have a few questions in this regard. 1) Where the log files are stored? Are they stored on DBFS?2) Can I read l...

  • 2332 Views
  • 3 replies
  • 2 kudos
Latest Reply
Anonymous
Not applicable
  • 2 kudos

Hi @Mohammad Saber​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Tha...

  • 2 kudos
2 More Replies
AmanSehgal
by Honored Contributor III
  • 1414 Views
  • 3 replies
  • 13 kudos

Resolved! Error on using Databricks Account API - invalid endpoint

I'm trying to configure audit logs for databricks and as a part of it I need to configure S3 storage using endpoint storage-configurations.But for some reason I'm getting invalid endpoint. I'm using following URL with basicAuth.https://accounts.cloud...

  • 1414 Views
  • 3 replies
  • 13 kudos
Latest Reply
Anonymous
Not applicable
  • 13 kudos

Hey there @Aman Sehgal​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you...

  • 13 kudos
2 More Replies
Labels