cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Kash
by Contributor III
  • 2776 Views
  • 6 replies
  • 7 kudos

Where is Alerts in the sidebar?

Hi everyone,I can't seem to find Alerts in the sidebar, also my data-explorer looks different from what I see in the videos. Do I need to upgrade my environment? Thanks,K

  • 2776 Views
  • 6 replies
  • 7 kudos
Latest Reply
Kash
Contributor III
  • 7 kudos

Hi group,After speaking with my rep, it appears that Databricks ALERTS is only for premium members even though that is not what is advertised on the site or in the documentation. This is unfortunate as data-quality is a concern for us and we don't fe...

  • 7 kudos
5 More Replies
chandan_a_v
by Valued Contributor
  • 10319 Views
  • 8 replies
  • 6 kudos

Resolved! logging.basicConfig not creating a file in Databricks

Hi,I am using the logger to log some parameters in my code and I want to save the file under DBFS. But for some reason the file is not getting created under DBFS. If I clear the state of the notebook and check the DBFS dir then file is present. Pleas...

  • 10319 Views
  • 8 replies
  • 6 kudos
Latest Reply
Anonymous
Not applicable
  • 6 kudos

Perhaps PyCharm sets a different working directory, meaning the file ends up in another place. Try providing a full path.

  • 6 kudos
7 More Replies
merca
by Valued Contributor II
  • 6420 Views
  • 6 replies
  • 3 kudos

⬆ Bump IPython to 7.31.1

Any plans to bump IPython version to 7.31.1 on the DBR 9.1 LTS runtime? If no other motivation

  • 6420 Views
  • 6 replies
  • 3 kudos
Latest Reply
merca
Valued Contributor II
  • 3 kudos

Hi @Kaniz Fatma​ !I just checked DBR 10.5 release notes and the IPython version listed there is 7.22.0 installed, but the version that is patched for this security issue is 7.31.1 Did I misread your last comment that it will be upgraded to safe versi...

  • 3 kudos
5 More Replies
GeorgeP
by New Contributor II
  • 1788 Views
  • 2 replies
  • 2 kudos

Errors when querying Azure DataBricks through DBeaver on macos

Configured DBeaver to work with either databricks latest driver or simba. I can connect and see databases, schemas, tables and columns. However, when a select statement is executed 30-40 seconds go by before I get the following error message: SQL...

  • 1788 Views
  • 2 replies
  • 2 kudos
Latest Reply
sage5616
Valued Contributor
  • 2 kudos

Has this issue been resolved? @aravhish solution did not help me. Any other options?I am experiencing the exact same issue with the same configuration on a Mac. Much help would be appreciated.

  • 2 kudos
1 More Replies
Ignacio33
by New Contributor II
  • 1481 Views
  • 2 replies
  • 1 kudos

"Backend services unavailable" when creating a default cluster

Hello all, I have been not using databricks CE for 4-5 months but today I've tried to create the default cluster, as always did, and I've got the error "Backend services unavailable". Is this a temporary problem or am I doing something wrong. Thanks ...

  • 1481 Views
  • 2 replies
  • 1 kudos
Latest Reply
tomasz
Databricks Employee
  • 1 kudos

There's currently an outage of the community edition. Please follow this link for status:https://status.databricks.com/pages/incident/5cf02dde58a00904bda41926/62cd7677db464e053416a89c

  • 1 kudos
1 More Replies
Judha2022
by New Contributor III
  • 2441 Views
  • 4 replies
  • 2 kudos
  • 2441 Views
  • 4 replies
  • 2 kudos
Latest Reply
Judha2022
New Contributor III
  • 2 kudos

Could you please let me know when it is available? It is critically important for me to get Databricks CE.Thanks again for your reply.

  • 2 kudos
3 More Replies
Mohit_m
by Valued Contributor II
  • 14083 Views
  • 1 replies
  • 1 kudos

Resolved! Job is failing with exception ClientAuthenticationError: DefaultAzureCredential failed to retrieve a token from the included credentials.

ClientAuthenticationError: DefaultAzureCredential failed to retrieve a token from the included credentials.Attempted credentials:EnvironmentCredential: EnvironmentCredential authentication unavailable. Environment variables are not fully configured.V...

  • 14083 Views
  • 1 replies
  • 1 kudos
Latest Reply
Mohit_m
Valued Contributor II
  • 1 kudos

Below docs are for reference:https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/identity/azure-identity/migration_guide.mdthere was a suggestion given to usefrom azure.common.credentials import ServicePrincipalCredentialsinstead offrom azure...

  • 1 kudos
brickster_2018
by Databricks Employee
  • 5607 Views
  • 2 replies
  • 0 kudos

Resolved! The job fails with HTTP 403

My jobs that are running for more than 48 hours are failing with HTTP 403 error

  • 5607 Views
  • 2 replies
  • 0 kudos
Latest Reply
willjoe
New Contributor III
  • 0 kudos

Check for URL errors and make sure you're specifying an actual web page file name and extension, not just a directory. Most websites are configured to disallow directory browsing, so a 403 Forbidden message when trying to display a folder instead of ...

  • 0 kudos
1 More Replies
chinoiserie
by New Contributor II
  • 5611 Views
  • 6 replies
  • 5 kudos

406 error code

https://databricks.com/try-databricks not acceptable, 406 error codeWhat should I do?

  • 5611 Views
  • 6 replies
  • 5 kudos
Latest Reply
willjoe
New Contributor III
  • 5 kudos

How Do I Fix HTTP 406 Not Acceptable?Troubleshooting on the Client-Side.Check Your URL. Rollback Recent Upgrades.Uninstall New Extensions, Modules, or Plugins.Check for Unexpected Database Changes. Troubleshooting on the Server-Side. Check the Config...

  • 5 kudos
5 More Replies
spdteng
by New Contributor III
  • 2143 Views
  • 2 replies
  • 3 kudos

Resolved! Voucher code error

Hi,I received a voucher in Nov 2020 for 'Databricks Certified Associate Developer for Apache Spark 3.0 exam' with an expiry date in Nov 2022. However I receive an error stating the promotion code is used. I have never used the code. Any idea who to c...

  • 2143 Views
  • 2 replies
  • 3 kudos
Latest Reply
willjoe
New Contributor III
  • 3 kudos

If your voucher code is not working, it could be due to the following reasons:The voucher code has expiredProducts in your shopping cart are not eligible for the discountTerms and conditions are not metIf applying a "new user only voucher" and our sy...

  • 3 kudos
1 More Replies
Atul_Sharan
by New Contributor II
  • 5856 Views
  • 3 replies
  • 3 kudos

Resolved! Error Code: 3206 - Processed HTTP request failed.

The ADF(Azur Data Factory) pipelines jobs executing several Databricks Notebook activities in parallel have been failing regularly with the following error "Error Code: 3206 - Processed HTTP request failed." The issue gets resolved on its own upon re...

  • 5856 Views
  • 3 replies
  • 3 kudos
Latest Reply
willjoe
New Contributor III
  • 3 kudos

Method 1 - Close Conflicting Programsdown voteWhen you get a runtime error, keep in mind that it is happening due to programs that are conflicting with each other. The first thing you can do to resolve the problem is to stop these conflicting program...

  • 3 kudos
2 More Replies
Pavan1
by New Contributor II
  • 2619 Views
  • 1 replies
  • 2 kudos

Does Spark MLlib support Generalized Additive Modeling? How does one go about implementing GAM models in Spark?

I want to implement GAM (Generalized additive model) model in Spark. Based on my research on online forums, I could not find the implementation of GAM models on Spark. Has anyone in this community attempted this? Does Spark MLlib support GAM?

  • 2619 Views
  • 1 replies
  • 2 kudos
Latest Reply
Pavan1
New Contributor II
  • 2 kudos

Hi @Kaniz Fatma​, thanks for sharing this. We ended up using pyGAM library in Python for this. This PDF is a good introduction. I will share my learnings once we complete our experiments.

  • 2 kudos
BradSheridan
by Valued Contributor
  • 2366 Views
  • 4 replies
  • 0 kudos

Resolved! Add custom tags to DLT cluster

We have a policy in our AWS account that whenever an EC2 instance is created, there are 5 mandatory tags that need to be added. When we create an All-Purpose cluster in the Databricks console, we can easily add these under Advance Options > Tags. How...

  • 2366 Views
  • 4 replies
  • 0 kudos
Latest Reply
BradSheridan
Valued Contributor
  • 0 kudos

@Tomasz Bacewicz​ Perfect!! worked like a charm...thank you!!do you think auto loader or DLT is a better approach to implementing a CDC pipeline in Databricks?

  • 0 kudos
3 More Replies

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group
Labels