- 2346 Views
- 2 replies
- 2 kudos
Resolved! Databricks Free Edition serverless
I am using the databricks free edition and want to learn how to use ML projects in databricks. However, when I try to connect to serverless, it does not allow me to do so. The only option I have is SOL compute. Is there a way to connect to serverless...
- 2346 Views
- 2 replies
- 2 kudos
- 2 kudos
@rc2 apologies, left you hanging on the last post. Was traveling back from the library.I imported this notebook from this resource: https://docs.databricks.com/aws/en/mlflow/end-to-end-example If you look at the navigation bar on the left hand side o...
- 2 kudos
- 1654 Views
- 2 replies
- 3 kudos
Resolved! Serving Endpoint: Container image creation
Hi TeamWhenever I try to create an endpoint from a model in Databricks, the process often gets stuck at the 'Container Image Creation' step. I've tried to understand what happens during this step, but couldn't find any detailed or helpful information...
- 1654 Views
- 2 replies
- 3 kudos
- 3284 Views
- 5 replies
- 3 kudos
Resolved! This API is disabled for users without the databricks-sql-access
Running a deply on github: Run databricks bundle deploydatabricks bundle deployshell: /usr/bin/bash -e {0}env:DATABRICKS_HOST: {{HOST}}DATABRICKS_CLIENT_ID: {{ID}}DATABRICKS_CLIENT_SECRET: ***DATABRICKS_BUNDLE_ENV: prodError: This API is disabled for...
- 3284 Views
- 5 replies
- 3 kudos
- 3 kudos
Got it working, yes I see it was a little confusing at first, the workspace displayed at the top right is the account information whereas the profile icon is where you can access the workspace settings. For anyone that got as confused as I did. Thank...
- 3 kudos
- 1784 Views
- 1 replies
- 1 kudos
The inference table is not updated
Hi, I am deploying a model with following code: w = WorkspaceClient()model_cfg = {"entity_name": uc_model,"entity_version": str(version),"workload_type": "CPU","workload_size": "Small","scale_to_zero_enabled": True}ai_gateway_config = AiGatewayConfig...
- 1784 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi Dharma, As mentioned in the documentation, Inference table log delivery is currently best effort, but logs are usually available within 1 hour of a request.Please try to query the inference tables after waiting for an hour. There are certain scena...
- 1 kudos
- 2177 Views
- 3 replies
- 5 kudos
Resolved! Not Able to run AutoML - RESOURCE DOES NOT EXIST ERROR
Hello,I'm new to both ML and Databricks. I'm running a Classification Experiment and getting a RESOURCE DOES NOT EXIST ERROR. It says the experiment_id does not exist. Can you help me point where to fix the error? I tried the Diagnose Error option, b...
- 2177 Views
- 3 replies
- 5 kudos
- 5 kudos
Hello Nt2good day!!If you view a stack trace and it looks similar to the following:RestException Traceback (most recent call last)File <command-XXXXXXXXXXXX>:72 mlflow.sklearn.autolog()...File /databricks/python/lib/python3.9/site-packages/mlflow/tra...
- 5 kudos
- 1026 Views
- 1 replies
- 1 kudos
Resolved! Model Inferencing
Any links, pointers to host a model in real time (similar to sagemaker endpoint in aws) - how can we host a model in DBX in real time - any documentation please?
- 1026 Views
- 1 replies
- 1 kudos
- 1 kudos
@Sachin_Amin you can find an example in our docs here: https://docs.databricks.com/aws/en/machine-learning/model-serving/model-serving-intro We also have free training courses on realtime model deployment for both classical ML (https://www.databricks...
- 1 kudos
- 1245 Views
- 1 replies
- 2 kudos
Resolved! How to choose legacy MLflow to upgrade Unity Catalog models
Hi there,I'm trying to upgrade MLflow models to Unity Catalog from legacy Models.I'm referencing this document. https://docs.databricks.com/aws/en/machine-learning/manage-model-lifecycle/upgrade-modelsBut I'm facing the error when I run `workspace_cl...
- 1245 Views
- 1 replies
- 2 kudos
- 2 kudos
I'm sorry, I figured it out myself.I have to write like this, workspace_client = MlflowClient(registry_uri="databricks")as the document: https://docs.databricks.com/gcp/en/machine-learning/manage-model-lifecycle/workspace-model-registry
- 2 kudos
- 3429 Views
- 5 replies
- 2 kudos
Python Debugger won't stop
I'm trying to use the debugger on python scripts that I'm running in my databricks workspace. The first few times i used it, it worked and stopped at my breakpoints. Since then, it just won't stop. I'm sure they are valid lines for breakpoints, but i...
- 3429 Views
- 5 replies
- 2 kudos
- 2 kudos
Thanks for the tips! This is a new cluster with DB rutime 16.4 LTS, access mode is "Dedicated". I have stopped and restarted the cluster several times and the same behavior persists. Does restarting the cluster not restart the Python kernel? If not t...
- 2 kudos
- 1582 Views
- 2 replies
- 0 kudos
DB vector search tutorial with GTE large - BPE tokenizer correct?
Hi, I was trying to implement a vector search use case based on the databricks example notebook with GTE large: https://docs.databricks.com/aws/en/notebooks/source/generative-ai/vector-search-foundation-embedding-model-gte-example.htmlFor chunking, t...
- 1582 Views
- 2 replies
- 0 kudos
- 0 kudos
I just found the definition and it is indeed word piece tokenization. So I think the tutorial is wrong. https://huggingface.co/Alibaba-NLP/gte-large-en-v1.5/blob/main/tokenizer.json
- 0 kudos
- 1852 Views
- 3 replies
- 0 kudos
Resolved! Full Memory Utilization
Hi Databricks Community. I need some suggestions on my issue. Basically we are using databricks asset bundle to deploy our forecasting repo and using aws nodes to run the forecast jobs. We built proper workflow.yml file to trigger the jobs.I am using...
- 1852 Views
- 3 replies
- 0 kudos
- 0 kudos
Hi @harishgehlot_03 Good day! May I know what the time was in the second case using a r6i.4xlarge instance type?
- 0 kudos
- 3555 Views
- 3 replies
- 1 kudos
Building a Data Quality pipeline with alerting
Hi there,My question is how do we setup a data-quality pipeline with alerting?Background: We would like to setup a data-quality pipeline to ensure the data we collect each day is consistent and complete. We will use key metrics found in our bronze JS...
- 3555 Views
- 3 replies
- 1 kudos
- 1 kudos
Hi Kash, on 4th point, do you guys have realtime ingestion to model ? or its batch. in case of batch, DLT will be fine i guess. but would love to know more. never seen realtime model updates ealier.
- 1 kudos
- 2283 Views
- 1 replies
- 0 kudos
Resolved! is a notebook available fro Advance machine Learning Operations
https://customer-academy.databricks.com/learn/courses/3508/advanced-machine-learning-operations/lessons/30967/automate-comprehensive-testingIs there a notebook that is accessible to follow along with the demo illustrated to setup pipeline with Git? o...
- 2283 Views
- 1 replies
- 0 kudos
- 0 kudos
With self-paced training you don't have access to the notebooks. You can purchase a subscription to Databricks Academy Labs (via Databricks Academy) for $200/year. This give you access to every lab we offer via paid training. Hope this helps, Lou. ...
- 0 kudos
- 946 Views
- 1 replies
- 0 kudos
ML for predictive maintenence use cases?!
How are you using ML to help determine predictive maintenance needs for your systems or operations?
- 946 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @LDogg, You can do predictive maintenance something like this:Start by streaming sensor or IoT data like temperature, pressure, vibration, etc. into Delta Lake using tools like Structured Streaming or Delta Live Tables.Next, we can process and eng...
- 0 kudos
- 6870 Views
- 6 replies
- 1 kudos
How can I use the feature store for time series out of sample prediction?
For instance, have a new model trained every Saturday with training data up to the previous Fri, and use such model to predict daily the following week?In the same context, if the features are keyed by date, could I create a training set with a diffe...
- 6870 Views
- 6 replies
- 1 kudos
- 1 kudos
Hello, I just came across this and I have a similar question. I am quite new to Databricks and the feature store, but I wanted to use it, however, I am having some difficulty figuring out what specifically I can do.In my case I am using XGBoost regre...
- 1 kudos
- 3473 Views
- 3 replies
- 0 kudos
spark_session invocation from executor side error, when using sparkXGBregressor and fe client
Hi I have created a model and pipeline using xgboost.spark's sparkXGBregressor and pyspark.ml's Pipeline instance. However, i run into a "RuntimeError: _get_spark_session should not be invoked from executor side." when i try to save the predictions i...
- 3473 Views
- 3 replies
- 0 kudos
- 0 kudos
I have tried the recommended solution logging the model with the parameter flavor = mlflow.pyfunc but it returns the following error when logging the model using FeatureEngineeringClient.log_model function:_validate_function_python_model(python_mode...
- 0 kudos
