Deploy tar.gz package from private git hub

[thushar]

We created Python package (.tar.gz) and kept it under private git.

We can able to connect to that git (using PAT) from the Azure databricks notebook.

Our requirement is to install that package from .tar.gz file for that notebook

"pip install https://USERNAME:PASWWORD@github.com/company_github/my_repo/my_package.tar.gz";

getting the below error.

does not appear to be a Python project: neither 'setup.py' nor 'pyproject.toml' found.

[Anonymous]

Hello @Thushar R​ - My name is Piper, and I'm a moderator for Databricks. It's great to have you here and thank you for your question. 🙂

Let's give the community a while to answer before we circle back around to this.

[Rahul_Samant · Accepted solution]

For installing the package using pip you need to package the repo using setup.py. check this link for more details https://packaging.python.org/en/latest/tutorials/packaging-projects/

alternatively you can pass the tar.gz using --py-files while submitting job.

[thushar]

Thanks, but why do I need to build again the package?

I am using VS Code for development, through VS Code I built my package and generated tar.gz and wheel file and committed in Git. So the aim is to install the already built package from Git to Azure databricks notebook.

What I understood from your reply is, even the package exists, we have to build it again for installing, so we need the setup.py file, right?

[Atanu]

If i understood correctly , yes that's correct you need to use the py file. is Rahul's solution worked @Thushar R​ ?