cancel
Showing results for 
Search instead for 
Did you mean: 
Machine Learning
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

db_noob
by New Contributor II
  • 1974 Views
  • 5 replies
  • 9 kudos

Azure - Databricks - account storage gen 2

Hello Every one, i am really new to databricks, just passed my apache developer certification on it.i also have a certification on data engineering with Azure.some fancy words here but i only started doing real deep work on them as i started a person...

  • 1974 Views
  • 5 replies
  • 9 kudos
Latest Reply
Debayan
Esteemed Contributor III
  • 9 kudos

Hi,If we go by the error , Invalid configuration value detected for fs.azure.account.keyStorage account access key to access data using the abfssprotocol cannot be used. Please refer this https://learn.microsoft.com/en-us/azure/databricks/storage/azu...

  • 9 kudos
4 More Replies
tmueller
by New Contributor III
  • 1408 Views
  • 4 replies
  • 6 kudos

Get mlflow to work with with custom Databricks docker container compute

We need conda for our python things, so i set up a Compute unit with a custom dockerfile. Now one of our ki engeniers tried to get mlflow working but it seems not connected to the mlflow of Databricks. How can i get mlflow in my container to work wit...

  • 1408 Views
  • 4 replies
  • 6 kudos
Latest Reply
Thijs
New Contributor III
  • 6 kudos

We have this same issues. Did you ever manage to solve this?

  • 6 kudos
3 More Replies
aranyics
by New Contributor
  • 496 Views
  • 1 replies
  • 1 kudos

Is it possible to start Databricks AutoML experiment remotely? (Azure Databricks)

Currently I am using Azure Machine Learning Studio for my work, and would like to compare performance of Azure and Databricks automl algorithms. Is it possible to write a notebook in Azure to start the automl algorithm in Databricks? My data is found...

  • 496 Views
  • 1 replies
  • 1 kudos
Latest Reply
Anonymous
Not applicable
  • 1 kudos

Hi @Csaba Aranyi​ Great to meet you, and thanks for your question! Let's see if your peers in the community have an answer to your question. Thanks.

  • 1 kudos
reachbharathan
by New Contributor III
  • 1665 Views
  • 4 replies
  • 5 kudos

Resolved! Authenticating gitlab with databricks via username & password?

Currently we have azure databricks and gitlab in our project, for integrating with code repository we have only gitlab, integrating with personal access token is possible,But it flagged out as potential risk of personal access token exposure, wanted...

  • 1665 Views
  • 4 replies
  • 5 kudos
Latest Reply
reachbharathan
New Contributor III
  • 5 kudos

Thank you folks,currently only way to integrate with gitlab is only with Personal Access Token,There is not way to intergrate gitlab via password, as per our security recommendation, we need to have additional mechanism to integrate as exposure of Pe...

  • 5 kudos
3 More Replies
Saurabh707344
by New Contributor III
  • 1630 Views
  • 4 replies
  • 3 kudos

Azure Databricks Capabilities: My objective is to evaluate Azure Databricks capability and Do I need to use Azure Devops or Jenkins or Databricks suffice the need.

hi,We have Real time streaming usecase where we have to build pipeline using Azure Databricks.My objective is to evaluate Azure Databricks capability and Do I need to use Azure Devops or Jenkins or Databricks suffice the need.Can you please provide c...

  • 1630 Views
  • 4 replies
  • 3 kudos
Latest Reply
StephanieRivera
Valued Contributor II
  • 3 kudos

I found these youtube videos to be beneficial. CI/CD with Azure Dev Ops Terraform Enablement - Part 1 of 2

  • 3 kudos
3 More Replies
Saurabh707344
by New Contributor III
  • 582 Views
  • 1 replies
  • 2 kudos

Sample Archietcture for Databricks MLOps

Do anyone have sample architectures for Mlops using Databricks and other possible variations of architecture ?

  • 582 Views
  • 1 replies
  • 2 kudos
Latest Reply
Annapurna_Hiriy
New Contributor III
  • 2 kudos

@Saurabh Singh​ This is well documented here:https://www.databricks.com/blog/2022/06/22/architecting-mlops-on-the-lakehouse.htmlPlease see: Reference architecture for MLOpsFurther refrences: Refer to The Big Book of MLOps for more discussion of the a...

  • 2 kudos
Saeid_H
by Contributor
  • 1946 Views
  • 2 replies
  • 0 kudos

Logging spark pipeline model using mlflow spark , leads to PythonSecurityException

Hello,I am currently using a simple pyspark pipeline to transform my training data, fit model and log the model using mlflow.spark. But I get this following error (with mlflow.sklearn it works perfectly fine but due to size of my data I need to use p...

  • 1946 Views
  • 2 replies
  • 0 kudos
Latest Reply
Anonymous
Not applicable
  • 0 kudos

Hi @Saeid Hedayati​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answer...

  • 0 kudos
1 More Replies
its-kumar
by New Contributor III
  • 3017 Views
  • 2 replies
  • 0 kudos

MLFlow Remote model registry connection is not working in Databricks

Dear community,I am having multiple Databricks workspaces in my azure subscription, and I have one central workspace. I want to use the central workspace for model registry and experiments tracking from the multiple other workspaces.So, If I am train...

  • 3017 Views
  • 2 replies
  • 0 kudos
Latest Reply
Anonymous
Not applicable
  • 0 kudos

@Kumar Shanu​ :The error you are seeing (API request to endpoint /api/2.0/mlflow/runs/create failed with error code 404 != 200) suggests that the API endpoint you are trying to access is not found. This could be due to several reasons, such as incorr...

  • 0 kudos
1 More Replies
Anonymous
by Not applicable
  • 573 Views
  • 2 replies
  • 3 kudos

www.databricks.com

Hello Dolly: Democratizing the magic of ChatGPT with open modelsDatabricks has just released a groundbreaking new blog post exploring ChatGPT, an open-source language model with the potential to transform the way we interact with technology. From cha...

  • 573 Views
  • 2 replies
  • 3 kudos
Latest Reply
Anonymous
Not applicable
  • 3 kudos

Lets get candid! Let me know your initial thoughts about LLM Models, ChatGpt, Dolly.

  • 3 kudos
1 More Replies
ajbush
by New Contributor III
  • 2485 Views
  • 5 replies
  • 3 kudos

Sample Datasets URL in Azure Databricks / access sample datasets when NPIP and Firewall is enabled

Hi,I have an Azure Databricks instance configured to use VNet injection with secure cluster connectivity. I have an Azure Firewall configured and controlling all traffic ingress and egress locations as per this article: https://learn.microsoft.com/en...

Screenshot 2023-02-08 at 4.45.47 PM
  • 2485 Views
  • 5 replies
  • 3 kudos
Latest Reply
Anonymous
Not applicable
  • 3 kudos

Hi @Alex Bush​ Hope everything is going great.Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us so we ...

  • 3 kudos
4 More Replies
Saeid_H
by Contributor
  • 3180 Views
  • 5 replies
  • 5 kudos

How to change the feature store delta table default path on DBFS?

Hi everyone,Would it be possible to change the default storage path of deature store, during creation and/or after creation? If you could also provide the python script to that I would appreciate. The current default path is:"dbfs/user/hive/warehouse...

  • 3180 Views
  • 5 replies
  • 5 kudos
Latest Reply
Anonymous
Not applicable
  • 5 kudos

Hi @Saeid Hedayati​ Hope everything is going great.Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us s...

  • 5 kudos
4 More Replies
Saeid_H
by Contributor
  • 4178 Views
  • 6 replies
  • 5 kudos

Register mlflow custom model, which has pickle files

Dear community,I want to basically store 2 pickle files during the training and model registry with my keras model. So that when I access the model from another workspace (using mlflow.set_registery_uri()) , these models can be accessed as well. The ...

  • 4178 Views
  • 6 replies
  • 5 kudos
Latest Reply
arzex
New Contributor II
  • 5 kudos

آموزش تولید محتوا

  • 5 kudos
5 More Replies
NSRBX
by Contributor
  • 2700 Views
  • 6 replies
  • 6 kudos

Resolved! Error loading model from mlflow: java.io.StreamCorruptedException: invalid type code: 00

Hello,I'm using, in my IDE, Databricks Connect version 9.1LTS ML to connect to a databricks cluster with spark version 3.1 and download a spark model that's been trained and saved using mlflow.So it seems like it's able to find a copy the model, but ...

  • 2700 Views
  • 6 replies
  • 6 kudos
Latest Reply
NSRBX
Contributor
  • 6 kudos

Hi @Kaniz Fatma​ and @Shanmugavel Chandrakasu​,It works after putting hadoop.dll into C:\Windows\System32 folder.I have hadoop version 3.3.1.I already had winutils.exe in the Hadoop bin folder.RegardsNath

  • 6 kudos
5 More Replies
User16461610613
by New Contributor II
  • 1245 Views
  • 1 replies
  • 2 kudos

Free Databricks Training on AWS, Azure, or Google Cloud Good news! You can now access free, in-depth Databricks training on AWS, Azure or Google Cloud...

Free Databricks Training on AWS, Azure, or Google CloudGood news! You can now access free, in-depth Databricks training on AWS, Azure or Google Cloud.  Our on-demand training series walks through how to:Streamline data ingest and management to build ...

image
  • 1245 Views
  • 1 replies
  • 2 kudos
Latest Reply
jose_gonzalez
Moderator
  • 2 kudos

Thank you for sharing this!!

  • 2 kudos
rubenteixeira
by New Contributor III
  • 1457 Views
  • 3 replies
  • 4 kudos

Enable programmatically writing to files

Hi everyone. I'm training a time series forecasting model in Azure Databricks. When I try to parallelize, it gives me this error:I have Contributor permission on the Databricks service, on the Azure Portal, and I'm an admin inside the Databricks work...

image.png
  • 1457 Views
  • 3 replies
  • 4 kudos
Latest Reply
jose_gonzalez
Moderator
  • 4 kudos

Hi @Rúben Teixeira​,Just a friendly follow-up. Did any of the responses help you to resolve your question? if it did, please mark it as best. Otherwise, please let us know if you still need help.

  • 4 kudos
2 More Replies
Labels