cancel
Showing results for 
Search instead for 
Did you mean: 
Machine Learning
Dive into the world of machine learning on the Databricks platform. Explore discussions on algorithms, model training, deployment, and more. Connect with ML enthusiasts and experts.
cancel
Showing results for 
Search instead for 
Did you mean: 
Data + AI Summit 2024 - Data Science & Machine Learning

Forum Posts

hawa
by Visitor
  • 50 Views
  • 1 replies
  • 0 kudos

Problem serving a langchain model on Databricks

Hi, I've encountered a problem of serving a langchain model I just created successfully on Databricks.I was using the following code to set up a model in unity catalog:from mlflow.models import infer_signatureimport mlflowimport langchainmlflow.set_r...

  • 50 Views
  • 1 replies
  • 0 kudos
Latest Reply
hawa
Visitor
  • 0 kudos

I suspected the issue is coming from this small error I got: Got error: Must specify a chain Type in config. I used the chain_type="stuff" when building the langchain but I'm not sure how to fix it.

  • 0 kudos
Steven_Roy
by New Contributor II
  • 135 Views
  • 1 replies
  • 0 kudos

Just Passed Databricks-Machine-Learning-Professional exam

Hi guys,I have the ML Professional exam scehduled for later this month and while I can find many resources, practice exams, and posts related to the ML Associate exam, I'm having trouble finding the same for the Professional exam.Anyone happen to hav...

  • 135 Views
  • 1 replies
  • 0 kudos
Latest Reply
jinn
New Contributor
  • 0 kudos

I got 90% on the Databricks-Machine-Learning-Professional exam, I am pleased with my results and thankful to this Site It provides premium quality service and has all the resources available. 

  • 0 kudos
avishkarborkar
by New Contributor
  • 483 Views
  • 6 replies
  • 4 kudos
  • 483 Views
  • 6 replies
  • 4 kudos
Latest Reply
szymon_dybczak
Contributor III
  • 4 kudos

There could be multiple reasone why you're getting this error @avishkarborkar . If the course you're following requires Unity Catalog, first you need to check if you have a premium workspace. Next you need to make sure that your workspace is enabled ...

  • 4 kudos
5 More Replies
Mikkel
by New Contributor
  • 66 Views
  • 1 replies
  • 0 kudos

Unable to Check Experiment Existence with path starting with /Workspace/ Directory in Databricks Pla

https://github.com/mlflow/mlflow/issues/11077  In Databricks, when attempting to set an experiment with an experiment_name specified as an absolute path from /Workspace/Shared/mlflow_experiment/<experiment_name>, the mlflow.set_experiment() function ...

  • 66 Views
  • 1 replies
  • 0 kudos
Latest Reply
Walter_C
Databricks Employee
  • 0 kudos

Before setting the experiment, use mlflow.get_experiment_by_name() to check if the experiment already exists. If it does, you can set the experiment without attempting to create it again.

  • 0 kudos
sergiopolimante
by New Contributor II
  • 118 Views
  • 1 replies
  • 0 kudos

What is the best to way to not deploy/run a workflow in production?

I am building and MLOps architecture.I do not want to deploy the training workflow to prod. My first approach was to selectively not deploy the workflow to prod, but this does not seem to be possible as in this thread:https://community.databricks.com...

  • 118 Views
  • 1 replies
  • 0 kudos
Latest Reply
Walter_C
Databricks Employee
  • 0 kudos

  Target Override Feature: You can use the target override feature to specify different configurations for different environments. However, this does not provide a direct way to exclude specific job resources.Environment-Specific Folders: Another app...

  • 0 kudos
DaPo
by New Contributor II
  • 91 Views
  • 1 replies
  • 1 kudos

Resolved! Using Datbricks Connect with serverless compute and MLflow

Hi all,I have been using databricks-connect with serverless compute to develop and debug my databricks related code. It worked great so far. Now I started integrating ML-Flow in my workflow, and I am encountering an issue. When I run the following co...

  • 91 Views
  • 1 replies
  • 1 kudos
Latest Reply
Walter_C
Databricks Employee
  • 1 kudos

The error you are encountering, pyspark.errors.exceptions.connect.AnalysisException: [CONFIG_NOT_AVAILABLE] Configuration spark.mlflow.modelRegistryUri is not available. SQLSTATE: 42K0I, is a known issue when using MLflow with serverless clusters in ...

  • 1 kudos
gustavocavsanto
by New Contributor
  • 125 Views
  • 1 replies
  • 0 kudos

Error 401: "Missing authorization details for accessing model serving endpoints" with OAuth Token on

I am trying to generate an OAuth token for my Azure Databricks workspace to access a model serving API in production. The code I’m using generates a token successfully, but I keep receiving a 401 error with the message "Missing authorization details ...

  • 125 Views
  • 1 replies
  • 0 kudos
Latest Reply
Walter_C
Databricks Employee
  • 0 kudos

You might need to follow steps in https://docs.databricks.com/en/machine-learning/model-serving/route-optimization.html#query-route-optimized-model-serving-endpoints on how to fetch an OAuth token programmatically

  • 0 kudos
rtreves
by New Contributor III
  • 2486 Views
  • 8 replies
  • 0 kudos

One-hot encoding of strong cardinality features failing, causes downstream issues

Hi Databricks support,I'm training an ML model using mlflow on DBR 13.3 LTS ML, Spark 3.4.1 using databricks.automl_runtime 0.2.17 and databricks.automl 1.20.3, with shap 0.45.1. My training data has two float-type columns with three or fewer unique ...

  • 2486 Views
  • 8 replies
  • 0 kudos
Latest Reply
NandiniN
Databricks Employee
  • 0 kudos

Hi @rtreves , sorry I was not able to investigate on the above. Not sure if you would be able to create a support ticket with Databricks as it may be an involved effort to review the code.  I do have a suggestion, instead of relying on the automatic ...

  • 0 kudos
7 More Replies
jonathanhodges
by New Contributor II
  • 228 Views
  • 1 replies
  • 0 kudos

Training Job Failure (Driver Error)

We have a new model training job that was running fine for a few days and then started failing. I have attached images for more details.I am wondering if 'can't reach driver cluster' is a red herring. It says the driver is healthy right before execut...

  • 228 Views
  • 1 replies
  • 0 kudos
Latest Reply
jonathanhodges
New Contributor II
  • 0 kudos

Sorry here is the full stack trace and one additional screen shot:ValueError: numpy.dtype size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObjectTraceback (most recent call last):File "/databricks/python_she...

  • 0 kudos
M_B
by New Contributor
  • 113 Views
  • 1 replies
  • 0 kudos

Serving model with custom scoring script to a real-time endpoint

Hi, new to databricks here and wasn't able to find relevant info in the documentation.Is it not possible to serve a model with a custom scoring script to an online endpoint on databricks to customise inference ? the customisation is related to incomi...

  • 113 Views
  • 1 replies
  • 0 kudos
Latest Reply
HaggMan
New Contributor II
  • 0 kudos

If I'm understanding, all you really want to do is have a pre/post - process function running with your model, is that correct? If so, you can do this by using the MLflow pyfunc model. Something  like they do here:https://docs.databricks.com/en/machi...

  • 0 kudos
ssequ
by New Contributor II
  • 85 Views
  • 0 replies
  • 0 kudos

Table-Model Lineage for models without online Feature Lookups

Hi community,I am looking for the recommended way to achieve table-model lineage in Unity Catalog for models that don't use Feature Lookups but only offline features. When I use FeatureEngineeringClient.create_training_set with feature_lookups + mlfl...

  • 85 Views
  • 0 replies
  • 0 kudos
pablobd
by Contributor II
  • 2071 Views
  • 3 replies
  • 1 kudos

Resolved! Model Serving Endpoint Creation through API

Hello,I am trying to create a model serving endpoint via the API as explained here: https://docs.databricks.com/api/workspace/servingendpoints/createI created a trusted IAM role with access to DynamoDB for the feature store. I try to use this field,"...

  • 2071 Views
  • 3 replies
  • 1 kudos
Latest Reply
malanevans
New Contributor II
  • 1 kudos

I have a similar issue with a model serving endpoint I created for serving external models (Amazon Bedrock) with Mosaic AI model serving (https://docs.databricks.com/en/generative-ai/external-models/index.html). For this also, I can use an instance p...

  • 1 kudos
2 More Replies
zed
by New Contributor II
  • 422 Views
  • 3 replies
  • 0 kudos

Consequences of Not Using write_table with Feature Engineering Client and INSERT OVERWRITE

Hello Databricks Community,I am currently using the Feature Engineering client and have a few questions about best practices for writing to Feature Store Tables.I would like to know more about not using the write_table method directly from the featur...

  • 422 Views
  • 3 replies
  • 0 kudos
Latest Reply
Brahmareddy
Valued Contributor III
  • 0 kudos

Hi @zed,How are you doing? As per my understanding, Consider using the write_table method from the Feature Engineering client to ensure that all Feature Store functionality is properly leveraged, such as cataloging, lineage tracking, and handling upd...

  • 0 kudos
2 More Replies

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group
Labels