cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

How to know, who started a job run?

MrJava
New Contributor III

Hi there!

We have different jobs/workflows configured in our Databricks workspace running on AWS and would like to know who actually started the job run? Are they started by a user or a service principle using curl?

Currently one can only see, who is the owner and who created the job config but not who triggerd an actual run.

I tried to find something in the audit logs we have activated for the jobs but was not successful.

Any hints are highly welcome!

14 REPLIES 14

Aviral-Bhardwaj
Esteemed Contributor III

check the event logs

and driver logs you will find that

AviralBhardwaj

MrJava
New Contributor III

Hi!

I checked the cluster logs and looked under "Event log". Under the Event type CREATING the listed user is not who actually started the job. The name that is listed there is always the user that is given under "Run as" in the job config.

Had a look into the driver logs but did not see any user information. Could you please be more specific on where to look there or give me a search pattern I can look for?

I would like to check who started the job, not under which user the cluster will run.

Aviral-Bhardwaj
Esteemed Contributor III

lets connect https://www.linkedin.com/in/aviralb/ to discuss more

AviralBhardwaj

Please keep the discussion on this platform so the whole community may participate.

Anonymous
Not applicable

Hi @Matthias Muellerโ€‹ 

Hope everything is going great.

Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us so we can help you. 

Cheers!

MrJava
New Contributor III

Nope, still no luck

AntonioAzure
New Contributor III

Trying to find exactly the same information. No luck and no entries in search engines so far. Looking for a solution for this.

JeffP
New Contributor II

Hi, were you able to find the information on who started a job run? I'm also searching for this information but was not able to find it. Thanks!

leonorgrosso
New Contributor II

I've just posted this idea on the Idea Portal of Databricks regarding this subject. Upvote it so it may get developed!

https://feedback.azure.com/d365community/idea/5d0fdbbf-eefb-ee11-a73c-0022485313bb

Harsha777
New Contributor III

Hi,
Any update on this feature?
With the info that is currently available, we cannot track the user who initiated the job run. The job run shows whether its scheduled or manual run but doesn't provide info on the user who triggered the manual run. "RUN AS" info is completely different which can be read from the job configuration itself as it is not a dynamic attribute. 

ahsan_aj
Contributor

It looks like there is no update on this, this is such a useful piece of information that ideally should have been available from the start.

Gianfranco
New Contributor II

It is possible to retrieve this information through the system tables here is an example

select *,user_identity.email
from system.access.audit
where event_date = 'xxx'
workspace_id = 'xxx'
and service_name = 'jobs' and request_params.job_id = 'xxx'

I added an additional filter to your query: action_name = "runNow" to retrieve the details. However, I believe the system tables only log events performed through the UI, and there are delays in when these logs appear. It would be beneficial to have this information available through an API for instant access.

hodb
New Contributor II

for some reason the user_identity.email includes only "unknown" or "Sustem-User"
any ideas how to repair to include the name of the person that triggered the job?

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโ€™t want to miss the chance to attend and share knowledge.

If there isnโ€™t a group near you, start one and help create a community that brings people together.

Request a New Group