cancel
Showing results for 
Search instead for 
Did you mean: 
Community Platform Discussions
Connect with fellow community members to discuss general topics related to the Databricks platform, industry trends, and best practices. Share experiences, ask questions, and foster collaboration within the community.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Rajaniesh
by New Contributor III
  • 19359 Views
  • 6 replies
  • 0 kudos

Error "Root storage credential for metastore does not exist"While creating the Databricks Volume in

Hi, I tried to create the databricks volume in unity catalog but it threw this error:Root storage credential for metastore XXXXXX does not exist. Please contact your Databricks representative or consider updating the metastore with a valid storage cr...

  • 19359 Views
  • 6 replies
  • 0 kudos
Latest Reply
KrzysztofPrzyso
New Contributor III
  • 0 kudos

I had a similar issue, that was resolved by assigning storage credential to the metastore using REST API call.- make sure that you are metastore or account admin- make sure that the storage credential is correctly configured, its access corrector ide...

  • 0 kudos
5 More Replies
Juanitoc10
by New Contributor II
  • 519 Views
  • 0 replies
  • 0 kudos

DBFS problem

Hi All, When trying to get to my DBFS in Databricks comunity edition I get the next errorI have already disabled and Enabled de DBFS file browser but it did not help

imagen.png
  • 519 Views
  • 0 replies
  • 0 kudos
danmlopsmaz
by New Contributor II
  • 1108 Views
  • 1 replies
  • 0 kudos

Issue with VSCode Extension and Databricks Cluster Using Docker Image

I've encountered a significant issue while using the VSCode extension for Databricks, particularly when working with a cluster configured with a Docker image. Here's a detailed description of the problem:Problem DescriptionWhen attempting to upload a...

danmlopsmaz_0-1721482196211.png danmlopsmaz_1-1721482508346.png
  • 1108 Views
  • 1 replies
  • 0 kudos
Latest Reply
danmlopsmaz
New Contributor II
  • 0 kudos

Hi @Retired_mod thanks for a such quick response.Actually, I am using the Dockerfile from the Databricks runtime example here: https://github.com/databricks/containers/blob/master/ubuntu/minimal/Dockerfile . The configuration with the VSCode extensio...

  • 0 kudos
sambgp
by New Contributor
  • 925 Views
  • 2 replies
  • 0 kudos

RDD Issue in Unity Catalog

 ################################################ New Code as per UC ################################################ def parse_json(df, *cols, clean=True        res = df    for i in cols:           if clean:            res = (                res.wit...

  • 925 Views
  • 2 replies
  • 0 kudos
Latest Reply
Rishabh_Tiwari
Databricks Employee
  • 0 kudos

Hi @sambgp , Thank you for reaching out to our community! We're here to help you.  To ensure we provide you with the best support, could you please take a moment to review the response and choose the one that best answers your question? Your feedback...

  • 0 kudos
1 More Replies
Krishna2110
by New Contributor II
  • 1003 Views
  • 4 replies
  • 2 kudos

Databricks All Delta Tables Data Read

If we want to read all the data of the databricks tables at single time how can we able to do it.

  • 1003 Views
  • 4 replies
  • 2 kudos
Latest Reply
szymon_dybczak
Contributor III
  • 2 kudos

Hi @Krishna2110 ,Here it is, it should work now tables = spark.sql("SHOW TABLES IN ewt_edp_prod.crm_raw").collect() for row in tables: table_name = f"ewt_edp_prod.{row[0]}.{row[1]}" try: df = spark.table(table_name) count = df...

  • 2 kudos
3 More Replies
Dp15
by Contributor
  • 2693 Views
  • 2 replies
  • 2 kudos

Resolved! Convert Date String to Date type Returning null

Hi,I am using Databricks SQL and I am converting a integer field which is of format ('20240719' or 'yyyyMMdd'),now I am able to convert it to date type using to_date(forecastedHorizonPeriodDate,'yyyyMMdd')I then tried to change the format of this dat...

  • 2693 Views
  • 2 replies
  • 2 kudos
Latest Reply
Dp15
Contributor
  • 2 kudos

Hey @szymon_dybczak Thank you for your answer

  • 2 kudos
1 More Replies
virendra1212
by New Contributor
  • 3730 Views
  • 1 replies
  • 0 kudos

Login issue

Hi,Getting below error message.We were not able to find a Community Edition workspace with this email. Please login to non-community-edition workspaces you may have access to.

  • 3730 Views
  • 1 replies
  • 0 kudos
Latest Reply
szymon_dybczak
Contributor III
  • 0 kudos

Hi @virendra1212 ,Take a look at below thread. Maybe some of the suggestions will help in your case.Community Edition Login Issues Below is a list of ... - Page 4 - Databricks Community - 26926You signed up before May 14, 2022 -Please confirm that th...

  • 0 kudos
payalbhatia
by New Contributor II
  • 578 Views
  • 0 replies
  • 0 kudos

Shuffle Partitions

What if I have lot of empty shuffled partitions due to data skewness Secondly , if the shuffle partition size is 128 MB and if the size of the key's partition is 700 MB 

  • 578 Views
  • 0 replies
  • 0 kudos
Junda
by New Contributor II
  • 1055 Views
  • 3 replies
  • 4 kudos

The way to visualize logging of Spark Job graphically

Is there any way to visualize logging such as execution runtime or memory usage of Spark Job graphically just like the image below by utilizing Databricks Partner Connect for free?Also, I'd appreciate to know any other ways to visualize the logging o...

  • 1055 Views
  • 3 replies
  • 4 kudos
Latest Reply
szymon_dybczak
Contributor III
  • 4 kudos

Pleasure to help

  • 4 kudos
2 More Replies
Leon_K
by New Contributor II
  • 3673 Views
  • 1 replies
  • 0 kudos

How to Add value to Comment Column in Databricks Catalog View for Foreign table (Azure SQL)

Hi.Struggling to add description by script to the comment column within the catalog view in Databricks, particularly for foreign/external tables sourced from Azure SQL.I have no issue doing that for delta tables. Also for information schema columns f...

  • 3673 Views
  • 1 replies
  • 0 kudos
Latest Reply
Leon_K
New Contributor II
  • 0 kudos

@Retired_mod Sorry, I did read this way back but forgot to reply.Thanks for information.And all those steps I found as well and for "3. Programmatic Approach (For Azure SQL)" if you are adding to sql any description that way I don't think it will app...

  • 0 kudos
samlexrod
by New Contributor III
  • 1469 Views
  • 2 replies
  • 0 kudos

Resolved! DLT Online Table with VNnet Enable on Blob Storage Get 403 Issue

I am trying to create an online table in a Unity catalog. However, I get a GET, 403 error.  DataPlaneException: Failed to start the DLT service on cluster . Please check the stack trace below or driver logs for more details. com.databricks.pipelines....

  • 1469 Views
  • 2 replies
  • 0 kudos
Latest Reply
samlexrod
New Contributor III
  • 0 kudos

I figured it out. It was because of the Network Connectivity Configurations. I did not have one setup with a private endpoint connection to the ADLS Gen2.  I followed the instructions here: https://learn.microsoft.com/en-us/azure/databricks/security/...

  • 0 kudos
1 More Replies
inagar
by New Contributor
  • 552 Views
  • 1 replies
  • 0 kudos

Unable to capture the Query result via JDBC client execution

As shown in below screenshots MERGE INTO command produces information about the result (num_affected_rows, num_updated_rows, num_deleted_rows, num_inserted_rows).Unable to get this information when the same query is being executed via JDBC client. Is...

Screenshot 2024-06-26 at 1.30.19 AM.png Screenshot 2024-06-26 at 1.36.10 AM (1).png Screenshot 2024-06-26 at 1.45.23 AM.png
  • 552 Views
  • 1 replies
  • 0 kudos
Latest Reply
Rishabh_Tiwari
Databricks Employee
  • 0 kudos

Hi @inagar , Thank you for reaching out to our community! We're here to help you.  To ensure we provide you with the best support, could you please take a moment to review the response and choose the one that best answers your question? Your feedback...

  • 0 kudos
inagar
by New Contributor
  • 533 Views
  • 1 replies
  • 0 kudos

Copying file from DBFS to a table of Databricks, Is there a way to get the errors at record level ?

We have file of data to be ingested into a table of Databricks. Following below approach,Uploaded file to DBFSCreating a temporary table and loading above file to the temporary table. CREATE TABLE [USING] Use MERGE INTO to merge temp_table created in...

  • 533 Views
  • 1 replies
  • 0 kudos
Latest Reply
Rishabh_Tiwari
Databricks Employee
  • 0 kudos

Hi @inagar , Thank you for reaching out to our community! We're here to help you.  To ensure we provide you with the best support, could you please take a moment to review the response and choose the one that best answers your question? Your feedback...

  • 0 kudos
inagar
by New Contributor
  • 528 Views
  • 2 replies
  • 0 kudos

Issues around float datatypes

Float data from file, is never getting copied to the temp_table created though the schema matches.As a workaround, using CREATE TABLE [USING] which is able to insert the file data to a temp_table. Is this a known issue? COPY INTO issue is explained w...

Screenshot 2024-06-21 at 10.45.02 PM.png Screenshot 2024-06-21 at 10.50.59 PM.png Screenshot 2024-06-21 at 10.53.57 PM.png Screenshot 2024-06-21 at 10.57.24 PM.png
  • 528 Views
  • 2 replies
  • 0 kudos
Latest Reply
Rishabh_Tiwari
Databricks Employee
  • 0 kudos

Hi @inagar , Thank you for reaching out to our community! We're here to help you.  To ensure we provide you with the best support, could you please take a moment to review the response and choose the one that best answers your question? Your feedback...

  • 0 kudos
1 More Replies
clock4eva
by New Contributor
  • 1328 Views
  • 2 replies
  • 2 kudos

Learning via Free Trial Azure

Hi I am following a Databricks course on Udemy and the course instructed to access Databricks via the free trial of Azure. Once I created my account on Azure and load Databricks, I try to create a cluster but this never succeeds. It takes an extremel...

  • 1328 Views
  • 2 replies
  • 2 kudos
Latest Reply
Rishabh_Tiwari
Databricks Employee
  • 2 kudos

Hi @clock4eva , Thank you for reaching out to our community! We're here to help you.  To ensure we provide you with the best support, could you please take a moment to review the response and choose the one that best answers your question? Your feedb...

  • 2 kudos
1 More Replies

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group
Top Kudoed Authors