cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Help
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Start a conversation
MrJava
by • New Contributor III
  • 9209 Views
  • 15 replies
  • 12 kudos

How to know, who started a job run?

Hi there!We have different jobs/workflows configured in our Databricks workspace running on AWS and would like to know who actually started the job run? Are they started by a user or a service principle using curl?Currently one can only see, who is t...

Data Engineering
Reply
  • 9209 Views
  • 15 replies
  • 12 kudos
View Replies
Latest Reply
mcveyroosevelt
New Contributor II
  • 12 kudos

To determine who started a job run in Databricks, you can use the Audit Logs feature by enabling workspace-level events and analyzing the runStarted events. Look for the userIdentity field within these logs, which identifies whether the run was trigg...

  • 12 kudos
14 More Replies
Ajay-Pandey
by • Esteemed Contributor III
  • 5316 Views
  • 5 replies
  • 0 kudos

How we can send databricks log to Azure Application Insight ?

Hi All,I want to send databricks logs to azure application insight.Is there any way we can do it ??Any blog or doc will help me.

Data Engineering
Reply
  • 5316 Views
  • 5 replies
  • 0 kudos
View Replies
Latest Reply
floringrigoriu
New Contributor II
  • 0 kudos

hi @Debayan in the  https://learn.microsoft.com/en-us/azure/architecture/databricks-monitoring/application-logs. there is a github repository mentioned https://github.com/mspnp/spark-monitoring ? That repository is marked as  maintainance mode.  Just...

  • 0 kudos
4 More Replies
Charmin
by • New Contributor
  • 1026 Views
  • 1 replies
  • 0 kudos

Why 'runCommand' action does NOT show up in databricksNotebook audit log table?

I understand databricks can send diagnostic/audit logs to log analytics in azure. There is a standard 'DatabricksNotebook' table that provides audit log for notebook actions. In this table there is an action called 'runCommand' but this does not show...

Data Engineering
Reply
  • 1026 Views
  • 1 replies
  • 0 kudos
View Replies
Latest Reply
rsamant07
New Contributor III
  • 0 kudos

Hi @Charmin patel​  , you need to enable verbose audit logging in workspace setting for runCommand to appear in the audit logs

  • 0 kudos
Mado
by • Valued Contributor II
  • 8483 Views
  • 4 replies
  • 3 kudos

Resolved! Databricks Audit Logs, What is "dataSourceId"?

Hi,I want to access the Databricks Audit Logs to check user activity.I created a Databricks workspace on the premium pricing tier.I configured Audit logs to be sent to Azure Diagnostic log delivery. What I got in the "Log Analytics Workspace":  I hav...

image image image
Data Engineering
Reply
  • 8483 Views
  • 4 replies
  • 3 kudos
View Replies
Latest Reply
youssefmrini
Databricks Employee
  • 3 kudos

The data_source_id field specifies the id of the SQL warehouse against which this query will run. You can use the Data Sources API to see a complete list of available SQL warehouses.

  • 3 kudos
3 More Replies
Mado
by • Valued Contributor II
  • 4066 Views
  • 3 replies
  • 2 kudos

Databricks Audit Logs, Where the log files are stored? How to read them?

Hi,I want to access the Databricks Audit Logs to check user activity. For example, the number of times that a table was viewed by a user.I have a few questions in this regard. 1) Where the log files are stored? Are they stored on DBFS?2) Can I read l...

Data Engineering
Reply
  • 4066 Views
  • 3 replies
  • 2 kudos
View Replies
Latest Reply
Anonymous
Not applicable
  • 2 kudos

Hi @Mohammad Saber​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Tha...

  • 2 kudos
2 More Replies
AmanSehgal
by • Honored Contributor III
  • 2786 Views
  • 2 replies
  • 12 kudos

Resolved! Error on using Databricks Account API - invalid endpoint

I'm trying to configure audit logs for databricks and as a part of it I need to configure S3 storage using endpoint storage-configurations.But for some reason I'm getting invalid endpoint. I'm using following URL with basicAuth.https://accounts.cloud...

Data Engineering
Reply
  • 2786 Views
  • 2 replies
  • 12 kudos
View Replies
Latest Reply
Anonymous
Not applicable
  • 12 kudos

Hey there @Aman Sehgal​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you...

  • 12 kudos
1 More Replies
Labels