topic Re: Databricks job with a type "Python wheel" does not work if "Package name" co in Administration & Architecture

Databricks job with a type "Python wheel" does not work if "Package name" contains dash

Sikalokym — Sun, 18 Feb 2024 10:54:54 GMT

Hello
I created a databricks job with a type "Python wheel". In the "Package name" field I assigned a python package which contains a dash in its name (see attach). The run of the job failed saying that could not import python package due to dash in the name of the python package.
How is it possible to create a job with a type "Python wheel" and a python package which contains dash in its name?

Re: Databricks job with a type "Python wheel" does not work if "Package name" co

feiyun0112 — Sun, 18 Feb 2024 13:47:32 GMT

Yes it is expected and documented behavior. Module names as
mentioned in import statements must be valid Python identifiers.
Hyphens are not allowed in identifiers.

Try renaming to test_package.

https://bugs.python.org/issue909226

Re: Databricks job with a type "Python wheel" does not work if "Package name" co

Sikalokym — Sun, 18 Feb 2024 22:31:55 GMT

Yes, it is known that hyphens are not allowed in python identifiers, while hyphens are allowed in python package names. There are a lot of python packages which contains hyphens in their name. In the configuration of a databricks job it is said that "Package name" should be a name as declared in package metadata (setup.py).
But, package name in metadata and a name of this package which we use for "Import" can be different, and this is normal behavior. So, it looks controversially - in general hyphens are allowed in package names, but in databricks job configuration hyphens are not allowed in package names. 🙂

Re: Databricks job with a type "Python wheel" does not work if "Package name" co

feiyun0112 — Tue, 20 Feb 2024 01:43:40 GMT

job use package name as module name, it is not allowed in python code

Re: Databricks job with a type "Python wheel" does not work if "Package name" co

AndréSalvati — Wed, 06 Mar 2024 16:48:51 GMT

There you can see a complete template project with a python wheel task and modules. Please, follow the instructions for deployment.

https://github.com/andre-salvati/databricks-template