Unable to run python script from git repo in Databricks job

[Divya_Bhadauria]

I'm getting cannot read python file on running this job which is configured to run a python script from git repo.

Run result unavailable: run failed with error message

Cannot read the python file /Repos/.internal/7c39d645692_commits/ff669d089cd8f93e98ccb96c513d5e675c2423f1/<Relative file path in git>

Please check driver logs for more details.

My question is why is it looking for the file internally (/Repos/.internal/) ?

[Anonymous]

@Divya Bhadauria​ :

The error message suggests that Databricks is trying to read the Python file from a path that starts with /Repos/.internal/. This indicates that the Databricks job is not able to download the Python file from the git repo specified in the job configuration.

To troubleshoot this issue, you can try the following steps:

1. Check that the git repo URL specified in the job configuration is correct and accessible. You can try to access the repo from your Databricks workspace to verify that it is accessible.
2. Check the branch, commit, or tag specified in the job configuration. Make sure that it corresponds to a valid version of the code that contains the Python file you are trying to execute.
3. Check the relative file path of the Python file specified in the job configuration. Make sure that it is correct and matches the actual file path in the git repo.
4. Check that the Databricks cluster has access to the internet and is able to connect to the git repo. You can try running a test command to download a file from the repo using a command like curl or wget
5. Check the driver logs for more information about the error. The driver logs may contain more detailed error messages or stack traces that can help identify the issue.

Once you have identified the issue, you can update the job configuration or make changes to the Databricks environment to resolve the issue.

[Anonymous]

Hi @Divya Bhadauria​ 

Hope everything is going great.

Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us so we can help you. 

Cheers!

[Divya_Bhadauria]

Hi Vidula,

Yes, the above solution worked out for me. Tried debugging using all of the above steps and it turned out the path I was using in the job config was incorrect.