cancel
Showing results for 
Search instead for 
Did you mean: 
Administration & Architecture
Explore discussions on Databricks administration, deployment strategies, and architectural best practices. Connect with administrators and architects to optimize your Databricks environment for performance, scalability, and security.
cancel
Showing results for 
Search instead for 
Did you mean: 

Databricks job with a type "Python wheel" does not work if "Package name" contains dash

Sikalokym
New Contributor II

Hello
I created a databricks job with a type "Python wheel". In the "Package name" field I assigned a python package which contains a dash in its name (see attach). The run of the job failed saying that could not import python package due to dash in the name of the python package.
How is it possible to create a job with a type "Python wheel" and a python package which contains dash in its name?

test_job.PNG

4 REPLIES 4

feiyun0112
Honored Contributor

Yes it is expected and documented behavior. Module names as
mentioned in import statements must be valid Python identifiers.
Hyphens are not allowed in identifiers.

Try renaming to test_package.

https://bugs.python.org/issue909226 

 

Yes, it is known that hyphens are not allowed in python identifiers, while hyphens are allowed in python package names. There are a lot of python packages which contains hyphens in their name. In the configuration of a databricks job it is said that "Package name" should be a name as declared in package metadata (setup.py).
But, package name in metadata and a name of this package which we use for "Import" can be different, and this is normal behavior. So, it looks controversially - in general hyphens are allowed in package names, but in databricks job configuration hyphens are not allowed in package names. 🙂 

feiyun0112
Honored Contributor

1.png

job use package name as module name, it is not allowed in python code 

AndréSalvati
New Contributor III

There you can see a complete template project with a python wheel task and modules. Please, follow the instructions for deployment.

https://github.com/andre-salvati/databricks-template

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group