cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Databricks - python error when importing wheel distribution package

Mohit_m
Valued Contributor II

In previous days all notebooks containing : 'import anomalydetection' worked just fine. There was no change in any configuration of the cluster, notebook or our imported library.

However recently notebooks just crashed with below error

Same happen also in the other clusters containing this library. 

Error:

TypeError: Descriptors cannot not be created directly. If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc bigger= 3.19.0. If you cannot immediately regenerate your protos, some other possible workarounds are: 1. Downgrade the protobuf package to 3.20.x or lower. 2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower).

As workaround i've set env variable PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python and started to work. As per recommendation this may slow down operations, what could the other solution?

1 ACCEPTED SOLUTION

Accepted Solutions

Mohit_m
Valued Contributor II

Solution: This is due to the latest version of protobuf library, please try to downgrade the library which should solve the issue

pip install protobuf==3.20.*

protobuf library versions which works: 3.20.1 if it does not work then try 3.18.1

View solution in original post

1 REPLY 1

Mohit_m
Valued Contributor II

Solution: This is due to the latest version of protobuf library, please try to downgrade the library which should solve the issue

pip install protobuf==3.20.*

protobuf library versions which works: 3.20.1 if it does not work then try 3.18.1

Join 100K+ Data Experts: Register Now & Grow with Us!

Excited to expand your horizons with us? Click here to Register and begin your journey to success!

Already a member? Login and join your local regional user group! If there isn’t one near you, fill out this form and we’ll create one for you to join!