cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Databricks - python error when importing wheel distribution package

Mohit_m
Valued Contributor II

In previous days all notebooks containing : 'import anomalydetection' worked just fine. There was no change in any configuration of the cluster, notebook or our imported library.

However recently notebooks just crashed with below error

Same happen also in the other clusters containing this library. 

Error:

TypeError: Descriptors cannot not be created directly. If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc bigger= 3.19.0. If you cannot immediately regenerate your protos, some other possible workarounds are: 1. Downgrade the protobuf package to 3.20.x or lower. 2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower).

As workaround i've set env variable PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python and started to work. As per recommendation this may slow down operations, what could the other solution?

1 ACCEPTED SOLUTION

Accepted Solutions

Mohit_m
Valued Contributor II

Solution: This is due to the latest version of protobuf library, please try to downgrade the library which should solve the issue

pip install protobuf==3.20.*

protobuf library versions which works: 3.20.1 if it does not work then try 3.18.1

View solution in original post

1 REPLY 1

Mohit_m
Valued Contributor II

Solution: This is due to the latest version of protobuf library, please try to downgrade the library which should solve the issue

pip install protobuf==3.20.*

protobuf library versions which works: 3.20.1 if it does not work then try 3.18.1

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group