cancel
Showing results for 
Search instead for 
Did you mean: 
Community Discussions
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

dofre
by New Contributor II
  • 234 Views
  • 1 replies
  • 0 kudos

Left Outer Join returns an Inner Join in Delta Live Tables

In our Delta Live Table pipeline I am simply joining two streaming tables to a new streaming table.We use the following code: @Dlt.create_table() def fact_event_faults(): events = dlt.read_stream('event_list').withWatermark('TimeStamp', '4 hours'...

4bba99f9-1293-42f9-ab53-f869af77a877.jpg
Community Discussions
Delta Live Table
structured streaming
  • 234 Views
  • 1 replies
  • 0 kudos
Latest Reply
Kaniz
Community Manager
  • 0 kudos

Hi @dofre, It appears that you’ve encountered an unexpected discrepancy in the results of your Delta Live Table pipeline. Let’s explore some potential reasons for this behavior: Streaming and Watermarking: In your pipeline, you’re working with s...

  • 0 kudos
phi_alpaca
by New Contributor III
  • 1119 Views
  • 9 replies
  • 6 kudos

Error at model serving for quantised models using bitsandbytes library

Hello,I've been trying to serve registered MLflow models at GPU Model Serving Endpoint, which works except for the models using bitsandbytes library. The library is used to quantise the LLM models into 4-bit/ 8-bit (e.g. Mistral-7B), however, it runs...

phi_alpaca_1-1708013174746.png
  • 1119 Views
  • 9 replies
  • 6 kudos
Latest Reply
G-M
Contributor
  • 6 kudos

@phi_alpacaWe have solved it by providing a conda_env.yaml when we log the model, all we needed was to add cudatoolkit=11.8 to the dependencies. 

  • 6 kudos
8 More Replies
violar
by New Contributor
  • 416 Views
  • 1 replies
  • 0 kudos

[Databricks][DatabricksJDBCDriver](500593) Communication link failure. Failed to connect to server.

I am using databricks jdbc driver to run a certain app. It runs fine for a few mins to hours and then I get the error [Databricks][DatabricksJDBCDriver](500593) Communication link failure. Failed to connect to server. Reason: HTTP Response code: 502,...

  • 416 Views
  • 1 replies
  • 0 kudos
Latest Reply
Kaniz
Community Manager
  • 0 kudos

Hi @violar, The error you’re encountering with the Databricks JDBC driver can be quite frustrating, but let’s troubleshoot it together. Here are a few suggestions to resolve this issue: Check Your Authentication Mechanism: The error message indi...

  • 0 kudos
thethirtyfour
by New Contributor III
  • 224 Views
  • 1 replies
  • 0 kudos

sparklyr::spark_read_csv forbidden 403 error

Hi,I am trying to read a csv file into a Spark DataFrame using sparklyr::spark_read_csv. I am receiving a 403 access denied error.I have stored my AWS credentials as environment variables, and can successfully read the file as an R dataframe using ar...

  • 224 Views
  • 1 replies
  • 0 kudos
Latest Reply
Kaniz
Community Manager
  • 0 kudos

Hi @thethirtyfour, It seems you’re encountering a 403 Forbidden error when trying to read a CSV file into a Spark DataFrame using sparklyr::spark_read_csv. Let’s troubleshoot this issue and explore potential solutions: IAM Roles vs. AWS Keys: En...

  • 0 kudos
yatharth
by New Contributor III
  • 294 Views
  • 2 replies
  • 0 kudos

Databricks Job cost(AWS)

Hi Databricks Community,I am looking for a formula/way to calculate the estimated cost for a job run, for which I have a few questions:1. Is there any formula to calculate the cost of any job like -> [(EC2 per hr cost) * (total time job ran)]and when...

  • 294 Views
  • 2 replies
  • 0 kudos
Latest Reply
yatharth
New Contributor III
  • 0 kudos

This looks a little bit confusing to me, I'm looking for a more straight forward answer, more like a simple formulaThanks though for your reply

  • 0 kudos
1 More Replies
Pratiksha1
by New Contributor
  • 219 Views
  • 1 replies
  • 0 kudos

Integrate databrick cluster with aws using cloudformation

Hello community, I am trying to launch databrick cluster using AWS  cloudformation template. I have checked public extension option in cloudformation and databrick cluster extension is also available in third party extension however not getting how t...

  • 219 Views
  • 1 replies
  • 0 kudos
Latest Reply
Kaniz
Community Manager
  • 0 kudos

Hi @Pratiksha1, When configuring a Databricks cluster using AWS CloudFormation, you can provide a JSON configuration to customize the cluster settings.    Let’s break down the steps:   Cluster Configuration in JSON: The JSON configuration allows you ...

  • 0 kudos
elgeo
by Valued Contributor II
  • 408 Views
  • 2 replies
  • 3 kudos

Hide widgets logic

Hello,We have recently created a notebook in order to allow users inserting/updating values in specific tables. The logic behind the update statements is included in a separate notebook where users don't have access. However we would like to know if ...

  • 408 Views
  • 2 replies
  • 3 kudos
Latest Reply
Kaniz
Community Manager
  • 3 kudos

Hey there! Thanks a bunch for being part of our awesome community!  We love having you around and appreciate all your questions. Take a moment to check out the responses – you'll find some great info. Your input is valuable, so pick the best solution...

  • 3 kudos
1 More Replies
FranPérez
by New Contributor III
  • 2411 Views
  • 2 replies
  • 1 kudos

Resolved! error loading databricks.proto

Hi,I'm using a DBR 13.3 LTS ML, and I want to set up a webhook trigger. I'm following the example notebook at https://learn.microsoft.com/en-us/azure/databricks/_extras/notebooks/source/mlflow/mlflow-model-registry-webhooks-python-client-example.html...

  • 2411 Views
  • 2 replies
  • 1 kudos
Latest Reply
StephanieRivera
Valued Contributor II
  • 1 kudos

Hi @FranPérez , The issue is that databricks-registry-webhooks has a databricks.proto file that collides with mlflow. Here is the fix: %pip install databricks-registry-webhooks mlflow==2.2.2 I also posted the fix on StackOverflow: https://stackoverfl...

  • 1 kudos
1 More Replies
Danny_Lee
by Contributor
  • 2342 Views
  • 2 replies
  • 0 kudos

My Gamification

Hi all,I've been training at https://partner-academy.databricks.com/ and I see this tab for My Gamification, however, whenever I open it it always says 0 badges, 0 points.  I have completed a number of courses, but there's no change.  Is this feature...

Danny_Lee_0-1708225241683.png
  • 2342 Views
  • 2 replies
  • 0 kudos
Latest Reply
Danny_Lee
Contributor
  • 0 kudos

Haven't received anything back on this and don't see any others with this issue, maybe its something on my side.  

  • 0 kudos
1 More Replies
146404
by New Contributor II
  • 329 Views
  • 1 replies
  • 0 kudos

workspace level sso gives authentication failed error

We have enabled workspace level sso , and have V2.0 version of databricks using azure EntraID groups and azure applications.Values in both databricks and azure application matchStill we get sso auth failed error.How can this be resolved , SAML tracer...

146404_0-1709016544478.png
  • 329 Views
  • 1 replies
  • 0 kudos
Latest Reply
Kaniz
Community Manager
  • 0 kudos

Hi @146404, To resolve the SSO authentication failed error in Databricks, ensure the SAML configuration in Databricks matches the Azure application settings, including the Issuer URL, Single Sign-On URL, and Entity ID. Check that the certificate for ...

  • 0 kudos
EvanMarth
by New Contributor III
  • 871 Views
  • 7 replies
  • 1 kudos

Cannot create an account to try Community Edition

Hi,Whenever I try to signup for an account, I keep getting the following message  - "an error has occurred. please try again later" when I click on the button "get started with databricks community edition".Could you please let me know why this could...

  • 871 Views
  • 7 replies
  • 1 kudos
Latest Reply
EvanMarth
New Contributor III
  • 1 kudos

Hello,Yeah I used this URL, I filled out my details and I selected : "Get Started With Community Edition"I did the puzzle and get the same error I had since the last few weeks : 'An error has occurred. Please try again later."And also I don't have an...

  • 1 kudos
6 More Replies
Milliman
by New Contributor
  • 277 Views
  • 1 replies
  • 0 kudos

How to add delay between databricks workflow job tasks.?

I want to add an explicit time delay between databricks workflow job tasks, any help would be greatly appreciated. Thanks

  • 277 Views
  • 1 replies
  • 0 kudos
Latest Reply
shan_chandra
Honored Contributor III
  • 0 kudos

@Milliman - you could add min_retry_interval_millis to add delay between start of the failed run and the subsequent retry run.  Reference is here

  • 0 kudos
Rajaniesh
by New Contributor III
  • 6088 Views
  • 5 replies
  • 0 kudos

Error "Root storage credential for metastore does not exist"While creating the Databricks Volume in

Hi, I tried to create the databricks volume in unity catalog but it threw this error:Root storage credential for metastore XXXXXX does not exist. Please contact your Databricks representative or consider updating the metastore with a valid storage cr...

  • 6088 Views
  • 5 replies
  • 0 kudos
Latest Reply
leandrochinelli
New Contributor II
  • 0 kudos

In this case, here's what's happening. Your metastore was created without a physical location, and when you don't specify a location for your catalog, by default it uses the metastore's location, which in this case doesn't exist. What I suggest you d...

  • 0 kudos
4 More Replies
ManiTejaG
by New Contributor II
  • 299 Views
  • 1 replies
  • 0 kudos

Workspace level changes in databricks

Hi Team,Is there any way in data bricks to track the changes that happen at notebook level. I know using system tables we can track the changes that happen at table/view/function/ other objects. However, is there any way to track all those changes in...

  • 299 Views
  • 1 replies
  • 0 kudos
Latest Reply
Ayushi_Suthar
Honored Contributor
  • 0 kudos

Hi @ManiTejaG , Hope you are doing well!  You can use the Audit logs to capture the notebook events that are logged at the workspace level. Please refer to this for more details : https://docs.databricks.com/en/administration-guide/account-settings/a...

  • 0 kudos
Priyam1
by New Contributor III
  • 911 Views
  • 4 replies
  • 1 kudos

Resolved! Unable to use the bamboolib in a databricks notebook

I am getting the same below error message while trying to use Bamboolib in a databricks notebook.Error: Could not display the widget. Please re-run the cell.I have installed the pypi libraries for bamboolib on the cluster and the databricks runtime v...

  • 911 Views
  • 4 replies
  • 1 kudos
Latest Reply
Priyam1
New Contributor III
  • 1 kudos

Thanks for the help.it worked.

  • 1 kudos
3 More Replies