cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

merca
by • Valued Contributor II
  • 22990 Views
  • 6 replies
  • 3 kudos

⬆ Bump IPython to 7.31.1

Any plans to bump IPython version to 7.31.1 on the DBR 9.1 LTS runtime? If no other motivation

  • 22990 Views
  • 6 replies
  • 3 kudos
Latest Reply
merca
Valued Contributor II
  • 3 kudos

Hi @Kaniz Fatma​ !I just checked DBR 10.5 release notes and the IPython version listed there is 7.22.0 installed, but the version that is patched for this security issue is 7.31.1 Did I misread your last comment that it will be upgraded to safe versi...

  • 3 kudos
5 More Replies
GeorgeP
by • New Contributor II
  • 3235 Views
  • 2 replies
  • 2 kudos

Errors when querying Azure DataBricks through DBeaver on macos

Configured DBeaver to work with either databricks latest driver or simba. I can connect and see databases, schemas, tables and columns. However, when a select statement is executed 30-40 seconds go by before I get the following error message: SQL...

  • 3235 Views
  • 2 replies
  • 2 kudos
Latest Reply
sage5616
Valued Contributor
  • 2 kudos

Has this issue been resolved? @aravhish solution did not help me. Any other options?I am experiencing the exact same issue with the same configuration on a Mac. Much help would be appreciated.

  • 2 kudos
1 More Replies
Ignacio33
by • New Contributor II
  • 2775 Views
  • 2 replies
  • 1 kudos

"Backend services unavailable" when creating a default cluster

Hello all, I have been not using databricks CE for 4-5 months but today I've tried to create the default cluster, as always did, and I've got the error "Backend services unavailable". Is this a temporary problem or am I doing something wrong. Thanks ...

  • 2775 Views
  • 2 replies
  • 1 kudos
Latest Reply
tomasz
Databricks Employee
  • 1 kudos

There's currently an outage of the community edition. Please follow this link for status:https://status.databricks.com/pages/incident/5cf02dde58a00904bda41926/62cd7677db464e053416a89c

  • 1 kudos
1 More Replies
Judha2022
by • New Contributor III
  • 4332 Views
  • 4 replies
  • 2 kudos
  • 4332 Views
  • 4 replies
  • 2 kudos
Latest Reply
Judha2022
New Contributor III
  • 2 kudos

Could you please let me know when it is available? It is critically important for me to get Databricks CE.Thanks again for your reply.

  • 2 kudos
3 More Replies
Mohit_m
by • Databricks Employee
  • 17075 Views
  • 1 replies
  • 2 kudos

Resolved! Job is failing with exception ClientAuthenticationError: DefaultAzureCredential failed to retrieve a token from the included credentials.

ClientAuthenticationError: DefaultAzureCredential failed to retrieve a token from the included credentials.Attempted credentials:EnvironmentCredential: EnvironmentCredential authentication unavailable. Environment variables are not fully configured.V...

  • 17075 Views
  • 1 replies
  • 2 kudos
Latest Reply
Mohit_m
Databricks Employee
  • 2 kudos

Below docs are for reference:https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/identity/azure-identity/migration_guide.mdthere was a suggestion given to usefrom azure.common.credentials import ServicePrincipalCredentialsinstead offrom azure...

  • 2 kudos
brickster_2018
by • Databricks Employee
  • 8240 Views
  • 2 replies
  • 0 kudos

Resolved! The job fails with HTTP 403

My jobs that are running for more than 48 hours are failing with HTTP 403 error

  • 8240 Views
  • 2 replies
  • 0 kudos
Latest Reply
willjoe
New Contributor III
  • 0 kudos

Check for URL errors and make sure you're specifying an actual web page file name and extension, not just a directory. Most websites are configured to disallow directory browsing, so a 403 Forbidden message when trying to display a folder instead of ...

  • 0 kudos
1 More Replies
chinoiserie
by • New Contributor II
  • 8109 Views
  • 6 replies
  • 5 kudos

406 error code

https://databricks.com/try-databricks not acceptable, 406 error codeWhat should I do?

  • 8109 Views
  • 6 replies
  • 5 kudos
Latest Reply
willjoe
New Contributor III
  • 5 kudos

How Do I Fix HTTP 406 Not Acceptable?Troubleshooting on the Client-Side.Check Your URL. Rollback Recent Upgrades.Uninstall New Extensions, Modules, or Plugins.Check for Unexpected Database Changes. Troubleshooting on the Server-Side. Check the Config...

  • 5 kudos
5 More Replies
spdteng
by • New Contributor III
  • 3809 Views
  • 2 replies
  • 3 kudos

Resolved! Voucher code error

Hi,I received a voucher in Nov 2020 for 'Databricks Certified Associate Developer for Apache Spark 3.0 exam' with an expiry date in Nov 2022. However I receive an error stating the promotion code is used. I have never used the code. Any idea who to c...

  • 3809 Views
  • 2 replies
  • 3 kudos
Latest Reply
willjoe
New Contributor III
  • 3 kudos

If your voucher code is not working, it could be due to the following reasons:The voucher code has expiredProducts in your shopping cart are not eligible for the discountTerms and conditions are not metIf applying a "new user only voucher" and our sy...

  • 3 kudos
1 More Replies
Atul_Sharan
by • New Contributor II
  • 8910 Views
  • 3 replies
  • 3 kudos

Resolved! Error Code: 3206 - Processed HTTP request failed.

The ADF(Azur Data Factory) pipelines jobs executing several Databricks Notebook activities in parallel have been failing regularly with the following error "Error Code: 3206 - Processed HTTP request failed." The issue gets resolved on its own upon re...

  • 8910 Views
  • 3 replies
  • 3 kudos
Latest Reply
willjoe
New Contributor III
  • 3 kudos

Method 1 - Close Conflicting Programsdown voteWhen you get a runtime error, keep in mind that it is happening due to programs that are conflicting with each other. The first thing you can do to resolve the problem is to stop these conflicting program...

  • 3 kudos
2 More Replies
Pavan1
by • New Contributor II
  • 4053 Views
  • 1 replies
  • 2 kudos

Does Spark MLlib support Generalized Additive Modeling? How does one go about implementing GAM models in Spark?

I want to implement GAM (Generalized additive model) model in Spark. Based on my research on online forums, I could not find the implementation of GAM models on Spark. Has anyone in this community attempted this? Does Spark MLlib support GAM?

  • 4053 Views
  • 1 replies
  • 2 kudos
Latest Reply
Pavan1
New Contributor II
  • 2 kudos

Hi @Kaniz Fatma​, thanks for sharing this. We ended up using pyGAM library in Python for this. This PDF is a good introduction. I will share my learnings once we complete our experiments.

  • 2 kudos
BradSheridan
by • Databricks Partner
  • 4557 Views
  • 4 replies
  • 0 kudos

Resolved! Add custom tags to DLT cluster

We have a policy in our AWS account that whenever an EC2 instance is created, there are 5 mandatory tags that need to be added. When we create an All-Purpose cluster in the Databricks console, we can easily add these under Advance Options > Tags. How...

  • 4557 Views
  • 4 replies
  • 0 kudos
Latest Reply
BradSheridan
Databricks Partner
  • 0 kudos

@Tomasz Bacewicz​ Perfect!! worked like a charm...thank you!!do you think auto loader or DLT is a better approach to implementing a CDC pipeline in Databricks?

  • 0 kudos
3 More Replies
abhay_sud
by • New Contributor II
  • 9207 Views
  • 3 replies
  • 3 kudos

Delta Live Table Pipeline Load Error

I am getting this error when running Delta Live Table Pipelines pulling from a source in our sandbox's local dbfs folder. It says the user is not authorized to perform this operation, whereas I am able to see the data when I run a simple select state...

image image image
  • 9207 Views
  • 3 replies
  • 3 kudos
Latest Reply
Anonymous
Not applicable
  • 3 kudos

Hey there @Abhay Sudhakaran​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution and mark an answer as best? Else please let us know if you need more help. We'd love to hear fr...

  • 3 kudos
2 More Replies
smanchem
by • New Contributor III
  • 4304 Views
  • 3 replies
  • 8 kudos

Resolved! AWS Private Link for DB workspace

What's the best way to set up a private link connection to Databricks without loosing the current settins? workspace

  • 4304 Views
  • 3 replies
  • 8 kudos
Latest Reply
Hubert-Dudek
Databricks MVP
  • 8 kudos

At the data and AI summit, there was a nice talk about AWS private link. Here is a link if you have access:https://dataaisummit.com/session-virtual/?v2477da705118cc74fd14460db021e1784e2eed5a7982c6482ec95cb2e86d259644b8741959f52a49e0e6908b82a9d860=C22...

  • 8 kudos
2 More Replies
Labels