cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Ignacio33
by New Contributor II
  • 1485 Views
  • 2 replies
  • 1 kudos

"Backend services unavailable" when creating a default cluster

Hello all, I have been not using databricks CE for 4-5 months but today I've tried to create the default cluster, as always did, and I've got the error "Backend services unavailable". Is this a temporary problem or am I doing something wrong. Thanks ...

  • 1485 Views
  • 2 replies
  • 1 kudos
Latest Reply
tomasz
Databricks Employee
  • 1 kudos

There's currently an outage of the community edition. Please follow this link for status:https://status.databricks.com/pages/incident/5cf02dde58a00904bda41926/62cd7677db464e053416a89c

  • 1 kudos
1 More Replies
Judha2022
by New Contributor III
  • 2448 Views
  • 4 replies
  • 2 kudos
  • 2448 Views
  • 4 replies
  • 2 kudos
Latest Reply
Judha2022
New Contributor III
  • 2 kudos

Could you please let me know when it is available? It is critically important for me to get Databricks CE.Thanks again for your reply.

  • 2 kudos
3 More Replies
Mohit_m
by Valued Contributor II
  • 14099 Views
  • 1 replies
  • 1 kudos

Resolved! Job is failing with exception ClientAuthenticationError: DefaultAzureCredential failed to retrieve a token from the included credentials.

ClientAuthenticationError: DefaultAzureCredential failed to retrieve a token from the included credentials.Attempted credentials:EnvironmentCredential: EnvironmentCredential authentication unavailable. Environment variables are not fully configured.V...

  • 14099 Views
  • 1 replies
  • 1 kudos
Latest Reply
Mohit_m
Valued Contributor II
  • 1 kudos

Below docs are for reference:https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/identity/azure-identity/migration_guide.mdthere was a suggestion given to usefrom azure.common.credentials import ServicePrincipalCredentialsinstead offrom azure...

  • 1 kudos
brickster_2018
by Databricks Employee
  • 5611 Views
  • 2 replies
  • 0 kudos

Resolved! The job fails with HTTP 403

My jobs that are running for more than 48 hours are failing with HTTP 403 error

  • 5611 Views
  • 2 replies
  • 0 kudos
Latest Reply
willjoe
New Contributor III
  • 0 kudos

Check for URL errors and make sure you're specifying an actual web page file name and extension, not just a directory. Most websites are configured to disallow directory browsing, so a 403 Forbidden message when trying to display a folder instead of ...

  • 0 kudos
1 More Replies
chinoiserie
by New Contributor II
  • 5613 Views
  • 6 replies
  • 5 kudos

406 error code

https://databricks.com/try-databricks not acceptable, 406 error codeWhat should I do?

  • 5613 Views
  • 6 replies
  • 5 kudos
Latest Reply
willjoe
New Contributor III
  • 5 kudos

How Do I Fix HTTP 406 Not Acceptable?Troubleshooting on the Client-Side.Check Your URL. Rollback Recent Upgrades.Uninstall New Extensions, Modules, or Plugins.Check for Unexpected Database Changes. Troubleshooting on the Server-Side. Check the Config...

  • 5 kudos
5 More Replies
spdteng
by New Contributor III
  • 2144 Views
  • 2 replies
  • 3 kudos

Resolved! Voucher code error

Hi,I received a voucher in Nov 2020 for 'Databricks Certified Associate Developer for Apache Spark 3.0 exam' with an expiry date in Nov 2022. However I receive an error stating the promotion code is used. I have never used the code. Any idea who to c...

  • 2144 Views
  • 2 replies
  • 3 kudos
Latest Reply
willjoe
New Contributor III
  • 3 kudos

If your voucher code is not working, it could be due to the following reasons:The voucher code has expiredProducts in your shopping cart are not eligible for the discountTerms and conditions are not metIf applying a "new user only voucher" and our sy...

  • 3 kudos
1 More Replies
Atul_Sharan
by New Contributor II
  • 5867 Views
  • 3 replies
  • 3 kudos

Resolved! Error Code: 3206 - Processed HTTP request failed.

The ADF(Azur Data Factory) pipelines jobs executing several Databricks Notebook activities in parallel have been failing regularly with the following error "Error Code: 3206 - Processed HTTP request failed." The issue gets resolved on its own upon re...

  • 5867 Views
  • 3 replies
  • 3 kudos
Latest Reply
willjoe
New Contributor III
  • 3 kudos

Method 1 - Close Conflicting Programsdown voteWhen you get a runtime error, keep in mind that it is happening due to programs that are conflicting with each other. The first thing you can do to resolve the problem is to stop these conflicting program...

  • 3 kudos
2 More Replies
Pavan1
by New Contributor II
  • 2625 Views
  • 1 replies
  • 2 kudos

Does Spark MLlib support Generalized Additive Modeling? How does one go about implementing GAM models in Spark?

I want to implement GAM (Generalized additive model) model in Spark. Based on my research on online forums, I could not find the implementation of GAM models on Spark. Has anyone in this community attempted this? Does Spark MLlib support GAM?

  • 2625 Views
  • 1 replies
  • 2 kudos
Latest Reply
Pavan1
New Contributor II
  • 2 kudos

Hi @Kaniz Fatma​, thanks for sharing this. We ended up using pyGAM library in Python for this. This PDF is a good introduction. I will share my learnings once we complete our experiments.

  • 2 kudos
BradSheridan
by Valued Contributor
  • 2374 Views
  • 4 replies
  • 0 kudos

Resolved! Add custom tags to DLT cluster

We have a policy in our AWS account that whenever an EC2 instance is created, there are 5 mandatory tags that need to be added. When we create an All-Purpose cluster in the Databricks console, we can easily add these under Advance Options > Tags. How...

  • 2374 Views
  • 4 replies
  • 0 kudos
Latest Reply
BradSheridan
Valued Contributor
  • 0 kudos

@Tomasz Bacewicz​ Perfect!! worked like a charm...thank you!!do you think auto loader or DLT is a better approach to implementing a CDC pipeline in Databricks?

  • 0 kudos
3 More Replies
abhay_sud
by New Contributor II
  • 5106 Views
  • 3 replies
  • 3 kudos

Delta Live Table Pipeline Load Error

I am getting this error when running Delta Live Table Pipelines pulling from a source in our sandbox's local dbfs folder. It says the user is not authorized to perform this operation, whereas I am able to see the data when I run a simple select state...

image image image
  • 5106 Views
  • 3 replies
  • 3 kudos
Latest Reply
Anonymous
Not applicable
  • 3 kudos

Hey there @Abhay Sudhakaran​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution and mark an answer as best? Else please let us know if you need more help. We'd love to hear fr...

  • 3 kudos
2 More Replies
smanchem
by New Contributor III
  • 2661 Views
  • 3 replies
  • 8 kudos

Resolved! AWS Private Link for DB workspace

What's the best way to set up a private link connection to Databricks without loosing the current settins? workspace

  • 2661 Views
  • 3 replies
  • 8 kudos
Latest Reply
Hubert-Dudek
Esteemed Contributor III
  • 8 kudos

At the data and AI summit, there was a nice talk about AWS private link. Here is a link if you have access:https://dataaisummit.com/session-virtual/?v2477da705118cc74fd14460db021e1784e2eed5a7982c6482ec95cb2e86d259644b8741959f52a49e0e6908b82a9d860=C22...

  • 8 kudos
2 More Replies
Kaher
by New Contributor
  • 3061 Views
  • 3 replies
  • 3 kudos
  • 3061 Views
  • 3 replies
  • 3 kudos
Latest Reply
Rheiman
Contributor II
  • 3 kudos

For general cluster decision making refer to this article https://docs.microsoft.com/en-gb/azure/databricks/clusters/cluster-config-best-practicesOnce you've selected a cluster that makes sense, run it and check your ganglia metrics to see whether yo...

  • 3 kudos
2 More Replies
Deepak_Goldwyn
by New Contributor III
  • 3428 Views
  • 5 replies
  • 2 kudos

Resolved! DLT Pipeline and Job Cluster

We have written few python functions(methods within a class) and packaged them as a wheel library.In the as-is situation we use to install that wheel library in All-Purpose cluster that we already have created. It works fine.In the to-be situtation(D...

  • 3428 Views
  • 5 replies
  • 2 kudos
Latest Reply
tomasz
Databricks Employee
  • 2 kudos

Does it give you an error when running the DLT pipeline specifically on the %pip command or does it not work in some other way? If it's the former, could you share the path format that you're using for the %pip command path?

  • 2 kudos
4 More Replies

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group
Labels