Databricks Community

Getting Py4J "Could not find py4j jar" error when trying to use pypmml, solution doesn't work

I'm trying to use pypmml in a DB notebook, but I'm getting the known `Error : Py4JError: Could not find py4j jar at` error. I've followed the solution here: https://kb.databricks.com/libraries/pypmml-fail-find-py4j-jar.html. However, this has not wor...

Is it possible to load MLFlow artifacts and models from local diretory to databricks DBFS?

I have been working locally and created a few models and now I want to move those to databricks/DBFS. Is it possible to do that?

ML Model serving cluster tags?

Is there a way to add tags automatically to ML Model serving clusters? I see we can add tags to the model itself which persist but any tags I add to the cluster serving it do not after the endpoint is stopped. This would be important to track billing...

Java Error for installation rasterframes

Hi all,I have followed the steps in this notebook to install rasterframes on my databricks cluster.Eventually I am able to import the following:from pyrasterframes import rf_ipython from pyrasterframes.utils import create_rf_spark_session from pyspar...

I should the same issue about this: https://community.databricks.com/s/topic/0TO3f000000CjVqGAK/py4jjavaerror

Could you pls help take a look at it? Thanks!

Model serving keep relaunching

Hello, I tried to serve my model realtime. Model process keeps relaunching.I am getting this error in the logs, TypeError: Descriptors cannot not be created directly. If this call came from a _pb2.py file, your generated code is out of date and must ...

How to perform time series forecasting for pharmaceuticals in Databricks?

Is there a latency difference between querying from the feature store and delta table?

Our team ran a benchmarking experiment comparing queries from the MLFlow feature store vs directly from delta table, and we found querying delta table was ~2-3 times faster than the feature store. So I’m wondering if someone else has done a similar b...

How to track features used and filters in MLFlow?

Hello everyone,We are experimenting with several approaches in a Machine Learning project ( binary classification), and we would like to keep track of those using MLFlow. We are using the feature store to build, store, and retrieve the features, and ...

Access multiple .mdb files using Python

Hi, I wanted to access multiple .mdb access files which are stored in the Azure Data Lake Storage(ADLS) or on Databricks File System using Python. Is it possible to guide me how can I achieve it? It would be great if you can share some code snippets ...

How do I install a non python/R/maven/jar library into a cluster?

I'm trying to install a non standard package into the cluster using the init scripts. The package I'm trying to install needs to be downloaded using wget, and uncompressed using tar. Then added to the PATH, or at least I need to know where the downlo...

Can't edit the cluster created by mlflow model serving

I'm trying to deploy a ml model into production using mlflow. while in that process, I have registered the model to mlflow models. After that it created the cluster but then it was in pending state forever. when I checked the model events, I see a pr...

Data model tool to connect to Databricks or Data lake?

Hi Everyone,From data modeling documentation (Dimensional/ ER Diagram), is there any tool available which can connect to databricks/ data lake and read the table structure directly and also updates the structure of table whenever there is a addition ...

MLFlow Project tutorial roadblock in viewing successful job run and run details. Resulted in failed status.

Hi! Any help will be greatly appreciated!! So I'm following this tutorial: https://docs.databricks.com/applications/mlflow/projects.html.I decided to use a folder in DBFS that contains my MLflow Project details. And so, in my project I have: MLprojec...