cancel
Showing results for 
Search instead for 
Did you mean: 
Administration & Architecture
cancel
Showing results for 
Search instead for 
Did you mean: 

Databricks job with a type "Python wheel" does not work if "Package name" contains dash

Sikalokym
New Contributor II

Hello
I created a databricks job with a type "Python wheel". In the "Package name" field I assigned a python package which contains a dash in its name (see attach). The run of the job failed saying that could not import python package due to dash in the name of the python package.
How is it possible to create a job with a type "Python wheel" and a python package which contains dash in its name?

test_job.PNG

4 REPLIES 4

feiyun0112
Contributor

Yes it is expected and documented behavior. Module names as
mentioned in import statements must be valid Python identifiers.
Hyphens are not allowed in identifiers.

Try renaming to test_package.

https://bugs.python.org/issue909226 

 

Yes, it is known that hyphens are not allowed in python identifiers, while hyphens are allowed in python package names. There are a lot of python packages which contains hyphens in their name. In the configuration of a databricks job it is said that "Package name" should be a name as declared in package metadata (setup.py).
But, package name in metadata and a name of this package which we use for "Import" can be different, and this is normal behavior. So, it looks controversially - in general hyphens are allowed in package names, but in databricks job configuration hyphens are not allowed in package names. 🙂 

feiyun0112
Contributor

1.png

job use package name as module name, it is not allowed in python code 

AndréSalvati
New Contributor III

There you can see a complete template project with a python wheel task and modules. Please, follow the instructions for deployment.

https://github.com/andre-salvati/databricks-template

Welcome to Databricks Community: Lets learn, network and celebrate together

Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections. 

Click here to register and join today! 

Engage in exciting technical discussions, join a group with your peers and meet our Featured Members.