- 32121 Views
- 3 replies
- 0 kudos
Resolved! Upload a file
Hi - I'm trying to upload a file, so that I can use the same in my notebook to try ML experiments with Databricks. From my workspace, I created a folder. But the option 'Create -> File' does not do anything. So not able to add any file. From a notebo...
- 32121 Views
- 3 replies
- 0 kudos
- 0 kudos
Can you try the approach mentioned in https://ganeshchandrasekaran.com/databricks-how-to-load-data-from-google-drive-github-c98d6b34d1b5
- 0 kudos
- 2341 Views
- 2 replies
- 0 kudos
Retrieve data from multiple .mdb files using Python.
Hello,I'm interested in accessing several .mdb Access files stored in either Azure Data Lake Storage (ADLS) or the Databricks File System using Python. Could you provide guidance on how to accomplish this? It would be immensely helpful if you could a...
- 2341 Views
- 2 replies
- 0 kudos
- 0 kudos
These are a couple of blogs and docs too https://docs.databricks.com/en/connect/storage/azure-storage.html
- 0 kudos
- 4968 Views
- 1 replies
- 0 kudos
UDF LLM DataBrick pickle error
Hi there,I am trying to parellize a text extraction via the Databrick foundational model.Any pointers to suggestions or examples are welcomeThe code and error below.model = "databricks-meta-llama-3-1-70b-instruct" temperature=0.0 max_tokens=1024 sch...
- 4968 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @llmnerd , Hope you are doing well! Upon reviewing the details provided, we have identified several observations regarding the SparkContext serialization error encountered. Please find a detailed analysis and our recommendations below:==== ANALYS...
- 0 kudos
- 6374 Views
- 2 replies
- 0 kudos
- 6374 Views
- 2 replies
- 0 kudos
- 0 kudos
I'm having the same problem but it's with the results of a query. Are there any fixes for this instance?
- 0 kudos
- 3955 Views
- 8 replies
- 4 kudos
`mlflow.login()` failed with error: Failed to validate databricks credentials: Basic Authentication
I have been using mlflow with Databricks community edition for 3 months without any issue. However, today when I tried to login to the mlflow host (https://community.cloud.databricks.com/), using this code, I keep getting the error below. # Setup MLf...
- 3955 Views
- 8 replies
- 4 kudos
- 4 kudos
Thank you for the update. It seems the provided reference https://docs.databricks.com/en/dev-tools/auth/oauth-u2m.html discusses authenticating access to the Databricks Platform (not the Databricks CE), where we need an Account ID to proceed with the...
- 4 kudos
- 3885 Views
- 2 replies
- 1 kudos
Resolved! Save model from AutoML to MLflow in LightGBM flavor
I want to get the LightGBM built-in variable importance values from a model that was generated by AutoML. That's not logged in the metrics by default - can I change a setting so that it will be logged?More fundamentally: what I'd really like is to ...
- 3885 Views
- 2 replies
- 1 kudos
- 1 kudos
Additional Considerations The pyfunc.add_to_model() function you mentioned is used to add the Python Function flavor to the model, which is different from changing the primary flavor of the logged model. That's why changing its parameter didn't solve...
- 1 kudos
- 8872 Views
- 1 replies
- 1 kudos
MlflowException: Unable to download model artifacts in Databricks while registering model with MLflo
I am attempting to log, register, and deploy a finetuned GPT2 model in Databricks. While I have been able to get my logging code to run, when I try to run my registration code, I get an MlflowException error.Here is my model logging code.mlflow.set_r...
- 8872 Views
- 1 replies
- 1 kudos
- 1 kudos
I've experience the same error. The issue is that the model uri is not correct.The model is logged with:mlflow.transformers.log_model( ... , artifact_path="gpt2", ...)The artifact_path is the last part of the model uri. If you don't specify it, it's ...
- 1 kudos
- 1342 Views
- 0 replies
- 0 kudos
Learn Databricks AI medium article series for fellow learners.
When it comes to machine learning, the platform plays a pivotal role in successful implementation. Databricks offers a best-in-class machine learning platform with cutting-edge features such as MLflow, Model Registry, Feature Store, and MLOps, which ...
- 1342 Views
- 0 replies
- 0 kudos
- 5027 Views
- 13 replies
- 2 kudos
MLflow: Connect python with Community Edition
Hello,I am new to databricks and want to work with MLFlow in the Databricks Community Edition. In python i am using mlflow.login(). This requests me to enter a password. But i do not have any password due to the fact that databricks login only requir...
- 5027 Views
- 13 replies
- 2 kudos
- 2 kudos
I am currently looking with our internal teams if this will be provided in the near future, still waiting for confirmation.
- 2 kudos
- 5005 Views
- 2 replies
- 2 kudos
Resolved! XGBoost Feature Weighting
We are trying to train a predictive ML model using the XGBoost Classifier. Part of the requirements we have gotten from our business team is to implement feature weighting as they have defined certain features mattering more than others. We have 69 f...
- 5005 Views
- 2 replies
- 2 kudos
- 2 kudos
Hello @sjohnston2 here is some information i found internally: Possible Causes Memory Access Issue: The segmentation fault suggests that the program is trying to access memory that it's not allowed to, which could be caused by an internal bug in XGBo...
- 2 kudos
- 2567 Views
- 2 replies
- 1 kudos
Resolved! Online Feature Table : Storage
Databricks Online Feature Table is in public preview , And we have some questions on this 1) What storage is being used for Online Feature table's Data. Our offline feature table is stored in Unity Catalog managed S3 bucket (Customer AWS ). Does onli...
- 2567 Views
- 2 replies
- 1 kudos
- 5163 Views
- 1 replies
- 1 kudos
Resolved! No Spark Session Available Within Model Serving Environment
Hi,Is it possible to have a Spark session, that can be to used query the Unity Catalog etc, available within a Model Serving?I have an MLFlow Pyfunc model that needs to get data from a Feature Table as part of its `.predict()` method. See my earlier ...
- 5163 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi @mharrison Creating a Spark session within a Model Serving environment is not directly supported, which is why you are encountering the Exception: No SparkSession Available! error. This limitation arises because the serving environment does not a...
- 1 kudos
- 27697 Views
- 3 replies
- 0 kudos
dbfs not found
Hi, I've saved a custom pyfunc and now I'm trying to load it in a pandas_udf. It works on small samples or if I repartition everything to 1 partition, but when I try to run it on a larger sample and distribute it across my cluster it fails repeatably...
- 27697 Views
- 3 replies
- 0 kudos
- 0 kudos
This problem can often be attributed to the model artifacts not being available on all the executors, especially in a distributed environment. Can you try using the dbutils.fs.refreshMounts() in your code? If the model is small enough, broadcast it t...
- 0 kudos
- 2078 Views
- 2 replies
- 0 kudos
Feature Lookup Help
Hi,ContextI'm looking for help trying to get Unity Catalog Feature Lookup to work with my model how I need it to.I have a trained darts time series model that takes as input to its `.predict()` method both the history of the variable in question, and...
- 2078 Views
- 2 replies
- 0 kudos
- 0 kudos
Thanks for your response. It sounds like the 2nd approach is best for me, modifying the `predict()` method to perform the required history lookup.Is it possible to do this via the Feature Engineering client within that method, or should I simply quer...
- 0 kudos
- 3371 Views
- 4 replies
- 1 kudos
How to serve a Unity Catalog ML model to external usage
Hello everyone I am following this notebook tutorial https://docs.databricks.com/en/machine-learning/manage-model-lifecycle/index.html#example-notebook Now I can register a machine learning model in Unity Catalog, but the tutorial only shows how to u...
- 3371 Views
- 4 replies
- 1 kudos
- 1 kudos
Hi @johndoe99012 If the answer resolved your question, please consider marking it as the solution. It helps others in the community find answers more easily.
- 1 kudos
