Can't run .py file using workflows anymore

[UdayPatel]

Dear all,

Greetings!

I have been trying to run a workflow job which runs successfully when a task is created using a Notebook file from a folder present in the Workspace but when the same task's type is changed to python script and a .py file is selected from the same folder, the workflow fails with the Resource Not Found error.  Both the files (notebook and .py) uses exactly same lines of code.

I have checked all the access rights from Workspace to File level, User and Groups, Cluster and Job level etc etc but can't find a single option to stop the job to execute a .py file but notebook at any level. The job is being run by a user which has full admin rights on the entire workspace.

Any help will be greatly appreciated!

Many thanks in advance!

Regards,

Uday

[AmanSehgal]

I just did a test and it worked.

Is it possible for you to provide a sample of your notebook code and python code.

Also, if you could please share your JOB details (JSON format should do)

[UdayPatel]

Hi Aman,

Thank you for your reply.

PFA the json, yaml and the error message with full details of the code etc. 

Regards,

Uday

[AmanSehgal]

Hi @UdayPatel 

Thank you for sharing the job details. 
I tried the same approach like running a notebook and then a python file. It worked for me successfully.

At python file level, could you please check the permissions and see if the user running the script has 'Can Manage' privilege?

[UdayPatel]

One update to this, when I removed the user from Admin group, the workflow could successfully run the .py file but when I re-added the user to Admin group, it started giving same error.. 😞 

In summary, a .py file owned by the user having Admin rights are not getting executed via Workflow but works fine when the admin rights are removed for that user.

[UdayPatel · Accepted solution]

Hi,

Have found the solution. It was due to following option being enabled under the Feature Enablement tab under  Databricks_Account_Console -- > Settings.

Thank you for all your help and the try!

Regards,

Uday