cancel
Showing results for 
Search instead for 
Did you mean: 
Machine Learning
Dive into the world of machine learning on the Databricks platform. Explore discussions on algorithms, model training, deployment, and more. Connect with ML enthusiasts and experts.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

db_noob
by New Contributor II
  • 3658 Views
  • 4 replies
  • 8 kudos

Azure - Databricks - account storage gen 2

Hello Every one, i am really new to databricks, just passed my apache developer certification on it.i also have a certification on data engineering with Azure.some fancy words here but i only started doing real deep work on them as i started a person...

  • 3658 Views
  • 4 replies
  • 8 kudos
Latest Reply
Debayan
Databricks Employee
  • 8 kudos

Hi,If we go by the error , Invalid configuration value detected for fs.azure.account.keyStorage account access key to access data using the abfssprotocol cannot be used. Please refer this https://learn.microsoft.com/en-us/azure/databricks/storage/azu...

  • 8 kudos
3 More Replies
tmueller
by New Contributor III
  • 2752 Views
  • 4 replies
  • 6 kudos

Get mlflow to work with with custom Databricks docker container compute

We need conda for our python things, so i set up a Compute unit with a custom dockerfile. Now one of our ki engeniers tried to get mlflow working but it seems not connected to the mlflow of Databricks. How can i get mlflow in my container to work wit...

  • 2752 Views
  • 4 replies
  • 6 kudos
Latest Reply
Thijs
New Contributor III
  • 6 kudos

We have this same issues. Did you ever manage to solve this?

  • 6 kudos
3 More Replies
aranyics
by New Contributor
  • 877 Views
  • 1 replies
  • 1 kudos

Is it possible to start Databricks AutoML experiment remotely? (Azure Databricks)

Currently I am using Azure Machine Learning Studio for my work, and would like to compare performance of Azure and Databricks automl algorithms. Is it possible to write a notebook in Azure to start the automl algorithm in Databricks? My data is found...

  • 877 Views
  • 1 replies
  • 1 kudos
Latest Reply
Anonymous
Not applicable
  • 1 kudos

Hi @Csaba Aranyi​ Great to meet you, and thanks for your question! Let's see if your peers in the community have an answer to your question. Thanks.

  • 1 kudos
reachbharathan
by New Contributor III
  • 2871 Views
  • 4 replies
  • 5 kudos

Resolved! Authenticating gitlab with databricks via username & password?

Currently we have azure databricks and gitlab in our project, for integrating with code repository we have only gitlab, integrating with personal access token is possible,But it flagged out as potential risk of personal access token exposure, wanted...

  • 2871 Views
  • 4 replies
  • 5 kudos
Latest Reply
reachbharathan
New Contributor III
  • 5 kudos

Thank you folks,currently only way to integrate with gitlab is only with Personal Access Token,There is not way to intergrate gitlab via password, as per our security recommendation, we need to have additional mechanism to integrate as exposure of Pe...

  • 5 kudos
3 More Replies
Saurabh707344
by New Contributor III
  • 2640 Views
  • 4 replies
  • 3 kudos

Azure Databricks Capabilities: My objective is to evaluate Azure Databricks capability and Do I need to use Azure Devops or Jenkins or Databricks suffice the need.

hi,We have Real time streaming usecase where we have to build pipeline using Azure Databricks.My objective is to evaluate Azure Databricks capability and Do I need to use Azure Devops or Jenkins or Databricks suffice the need.Can you please provide c...

  • 2640 Views
  • 4 replies
  • 3 kudos
Latest Reply
StephanieAlba
Databricks Employee
  • 3 kudos

I found these youtube videos to be beneficial. CI/CD with Azure Dev Ops Terraform Enablement - Part 1 of 2

  • 3 kudos
3 More Replies
Saurabh707344
by New Contributor III
  • 1022 Views
  • 1 replies
  • 2 kudos

Sample Archietcture for Databricks MLOps

Do anyone have sample architectures for Mlops using Databricks and other possible variations of architecture ?

  • 1022 Views
  • 1 replies
  • 2 kudos
Latest Reply
Annapurna_Hiriy
Databricks Employee
  • 2 kudos

@Saurabh Singh​ This is well documented here:https://www.databricks.com/blog/2022/06/22/architecting-mlops-on-the-lakehouse.htmlPlease see: Reference architecture for MLOpsFurther refrences: Refer to The Big Book of MLOps for more discussion of the a...

  • 2 kudos
Saeid_H
by Contributor
  • 3263 Views
  • 2 replies
  • 0 kudos

Logging spark pipeline model using mlflow spark , leads to PythonSecurityException

Hello,I am currently using a simple pyspark pipeline to transform my training data, fit model and log the model using mlflow.spark. But I get this following error (with mlflow.sklearn it works perfectly fine but due to size of my data I need to use p...

  • 3263 Views
  • 2 replies
  • 0 kudos
Latest Reply
Anonymous
Not applicable
  • 0 kudos

Hi @Saeid Hedayati​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answer...

  • 0 kudos
1 More Replies
its-kumar
by New Contributor III
  • 9030 Views
  • 2 replies
  • 0 kudos

MLFlow Remote model registry connection is not working in Databricks

Dear community,I am having multiple Databricks workspaces in my azure subscription, and I have one central workspace. I want to use the central workspace for model registry and experiments tracking from the multiple other workspaces.So, If I am train...

  • 9030 Views
  • 2 replies
  • 0 kudos
Latest Reply
Anonymous
Not applicable
  • 0 kudos

@Kumar Shanu​ :The error you are seeing (API request to endpoint /api/2.0/mlflow/runs/create failed with error code 404 != 200) suggests that the API endpoint you are trying to access is not found. This could be due to several reasons, such as incorr...

  • 0 kudos
1 More Replies
Anonymous
by Not applicable
  • 1210 Views
  • 2 replies
  • 3 kudos

www.databricks.com

Hello Dolly: Democratizing the magic of ChatGPT with open modelsDatabricks has just released a groundbreaking new blog post exploring ChatGPT, an open-source language model with the potential to transform the way we interact with technology. From cha...

  • 1210 Views
  • 2 replies
  • 3 kudos
Latest Reply
Anonymous
Not applicable
  • 3 kudos

Lets get candid! Let me know your initial thoughts about LLM Models, ChatGpt, Dolly.

  • 3 kudos
1 More Replies
ajbush
by New Contributor III
  • 4646 Views
  • 5 replies
  • 3 kudos

Sample Datasets URL in Azure Databricks / access sample datasets when NPIP and Firewall is enabled

Hi,I have an Azure Databricks instance configured to use VNet injection with secure cluster connectivity. I have an Azure Firewall configured and controlling all traffic ingress and egress locations as per this article: https://learn.microsoft.com/en...

Screenshot 2023-02-08 at 4.45.47 PM
  • 4646 Views
  • 5 replies
  • 3 kudos
Latest Reply
Anonymous
Not applicable
  • 3 kudos

Hi @Alex Bush​ Hope everything is going great.Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us so we ...

  • 3 kudos
4 More Replies
Saeid_H
by Contributor
  • 4765 Views
  • 5 replies
  • 5 kudos

How to change the feature store delta table default path on DBFS?

Hi everyone,Would it be possible to change the default storage path of deature store, during creation and/or after creation? If you could also provide the python script to that I would appreciate. The current default path is:"dbfs/user/hive/warehouse...

  • 4765 Views
  • 5 replies
  • 5 kudos
Latest Reply
Anonymous
Not applicable
  • 5 kudos

Hi @Saeid Hedayati​ Hope everything is going great.Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us s...

  • 5 kudos
4 More Replies
Saeid_H
by Contributor
  • 11026 Views
  • 5 replies
  • 4 kudos

Register mlflow custom model, which has pickle files

Dear community,I want to basically store 2 pickle files during the training and model registry with my keras model. So that when I access the model from another workspace (using mlflow.set_registery_uri()) , these models can be accessed as well. The ...

  • 11026 Views
  • 5 replies
  • 4 kudos
Latest Reply
arzex
New Contributor II
  • 4 kudos

آموزش تولید محتوا

  • 4 kudos
4 More Replies
NSRBX
by Contributor
  • 4234 Views
  • 4 replies
  • 5 kudos

Resolved! Error loading model from mlflow: java.io.StreamCorruptedException: invalid type code: 00

Hello,I'm using, in my IDE, Databricks Connect version 9.1LTS ML to connect to a databricks cluster with spark version 3.1 and download a spark model that's been trained and saved using mlflow.So it seems like it's able to find a copy the model, but ...

  • 4234 Views
  • 4 replies
  • 5 kudos
Latest Reply
NSRBX
Contributor
  • 5 kudos

Hi @Kaniz Fatma​ and @Shanmugavel Chandrakasu​,It works after putting hadoop.dll into C:\Windows\System32 folder.I have hadoop version 3.3.1.I already had winutils.exe in the Hadoop bin folder.RegardsNath

  • 5 kudos
3 More Replies
User16461610613
by New Contributor II
  • 2027 Views
  • 1 replies
  • 2 kudos

Free Databricks Training on AWS, Azure, or Google Cloud Good news! You can now access free, in-depth Databricks training on AWS, Azure or Google Cloud...

Free Databricks Training on AWS, Azure, or Google CloudGood news! You can now access free, in-depth Databricks training on AWS, Azure or Google Cloud.  Our on-demand training series walks through how to:Streamline data ingest and management to build ...

image
  • 2027 Views
  • 1 replies
  • 2 kudos
Latest Reply
jose_gonzalez
Databricks Employee
  • 2 kudos

Thank you for sharing this!!

  • 2 kudos
rubenteixeira
by New Contributor III
  • 2504 Views
  • 3 replies
  • 4 kudos

Enable programmatically writing to files

Hi everyone. I'm training a time series forecasting model in Azure Databricks. When I try to parallelize, it gives me this error:I have Contributor permission on the Databricks service, on the Azure Portal, and I'm an admin inside the Databricks work...

image.png
  • 2504 Views
  • 3 replies
  • 4 kudos
Latest Reply
jose_gonzalez
Databricks Employee
  • 4 kudos

Hi @Rúben Teixeira​,Just a friendly follow-up. Did any of the responses help you to resolve your question? if it did, please mark it as best. Otherwise, please let us know if you still need help.

  • 4 kudos
2 More Replies
Labels