topic Re: Can't run .py file using workflows anymore in Get Started Discussions

Can't run .py file using workflows anymore

UdayPatel — Sun, 21 Apr 2024 19:41:12 GMT

Dear all,

Greetings!

I have been trying to run a workflow job which runs successfully when a task is created using a Notebook file from a folder present in the Workspace but when the same task's type is changed to python script and a .py file is selected from the same folder, the workflow fails with the Resource Not Found error. Both the files (notebook and .py) uses exactly same lines of code.

I have checked all the access rights from Workspace to File level, User and Groups, Cluster and Job level etc etc but can't find a single option to stop the job to execute a .py file but notebook at any level. The job is being run by a user which has full admin rights on the entire workspace.

Any help will be greatly appreciated!

Many thanks in advance!

Regards,

Uday

Re: Can't run .py file using workflows anymore

AmanSehgal — Mon, 22 Apr 2024 00:55:16 GMT

I just did a test and it worked.

Is it possible for you to provide a sample of your notebook code and python code.

Also, if you could please share your JOB details (JSON format should do)

Re: Can't run .py file using workflows anymore

UdayPatel — Mon, 22 Apr 2024 09:26:59 GMT

Hi Aman,

Thank you for your reply.

PFA the json, yaml and the error message with full details of the code etc.

Regards,

Uday

Re: Can't run .py file using workflows anymore

UdayPatel — Mon, 22 Apr 2024 13:31:52 GMT

One update to this, when I removed the user from Admin group, the workflow could successfully run the .py file but when I re-added the user to Admin group, it started giving same error.. 😞

In summary, a .py file owned by the user having Admin rights are not getting executed via Workflow but works fine when the admin rights are removed for that user.

Re: Can't run .py file using workflows anymore

AmanSehgal — Mon, 22 Apr 2024 13:52:15 GMT

Hi @UdayPatel

Thank you for sharing the job details.
I tried the same approach like running a notebook and then a python file. It worked for me successfully.

At python file level, could you please check the permissions and see if the user running the script has 'Can Manage' privilege?

Re: Can't run .py file using workflows anymore

UdayPatel — Mon, 22 Apr 2024 14:36:52 GMT

Hi,

Have found the solution. It was due to following option being enabled under the Feature Enablement tab under Databricks_Account_Console -- > Settings.

Thank you for all your help and the try!

Regards,

Uday