cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Help
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Start a conversation
grazie
by • Contributor
  • 2514 Views
  • 2 replies
  • 0 kudos

Resolved! slack notification (webhook) failing

POST to the slack webhook from local http client works_"Test" action on "System Notifcations" page in Databricks gives response 400 bad request

Data Engineering
Reply
  • 2514 Views
  • 2 replies
  • 0 kudos
View Replies
Latest Reply
grazie
Contributor
  • 0 kudos

All in all, I got confused by a misconfiguration.A notification was setup as a webhook notificaton instead of a slack notification by mistake in a particular workspace, which caused the confusion. So, "problem in chair, not computer". If anything, it...

  • 0 kudos
1 More Replies
DianGermishuiz1
by • New Contributor III
  • 7666 Views
  • 7 replies
  • 4 kudos

Resolved! Unsupported cloud file system schema 'wasbs' error in custom catalog, but not in hive_metastore

When trying to run a create table USING CSV that pulls data from an azure blob storage path in a custom catalog I created, I get an error stating Unsupported cloud file system schema 'wasbs'.However, when I run this code in the hive_metastore catalog...

wasbs cloud path unsupported error
Data Engineering
Reply
  • 7666 Views
  • 7 replies
  • 4 kudos
View Replies
Latest Reply
youssefmrini
Databricks Employee
  • 4 kudos

WASB Has been deprecated.

  • 4 kudos
6 More Replies
jomt
by • New Contributor III
  • 2845 Views
  • 3 replies
  • 1 kudos

Error in SQL Warehouse: User is not part of org

I tried to start the Databricks SQL Warehouse cluster today, but received the following error message:Clusters are failing to launch. Cluster launch will be retired Request to create a cluster failed with an exception: PERMISSION_DENIED: User xxxx is...

Data Engineering
Reply
  • 2845 Views
  • 3 replies
  • 1 kudos
View Replies
Latest Reply
jose_gonzalez
Databricks Employee
  • 1 kudos

Hi @Joakim Torsvik​,Just a friendly follow-up. Did any of the responses help you to resolve your question? if it did, please mark it as best. Otherwise, please let us know if you still need help.

  • 1 kudos
2 More Replies
Databrickguy
by • New Contributor II
  • 2720 Views
  • 2 replies
  • 2 kudos

How to check/list the tables which has CDF enabled?

How to list all the tables which has CDF enabled?I can review a table to find out if CDF is enable with below code.SHOW TBLPROPERTIES tableA(delta.enableChangeDataFeed)The return key ...

Data Engineering
Reply
  • 2720 Views
  • 2 replies
  • 2 kudos
View Replies
Latest Reply
jose_gonzalez
Databricks Employee
  • 2 kudos

Hhi @Tim zhang​,Just a friendly follow-up. Did any of the responses help you to resolve your question? if it did, please mark it as best. Otherwise, please let us know if you still need help.

  • 2 kudos
1 More Replies
amitca71
by • Contributor II
  • 3296 Views
  • 6 replies
  • 0 kudos

Customer-managed VPC with empty default security group

Hi,I use self managed VPC. when i use security group the jobs clusters are being created.if i make the default security group empty and create customized security group (required by soc2 to have empty default), with same definition of ingress (allow...

Data Engineering
Reply
  • 3296 Views
  • 6 replies
  • 0 kudos
View Replies
Latest Reply
jose_gonzalez
Databricks Employee
  • 0 kudos

Hi @Amit Cahanovich​,Just a friendly follow-up. Did any of the responses help you to resolve your question? if it did, please mark it as best. Otherwise, please let us know if you still need help.

  • 0 kudos
5 More Replies
eacuna
by • New Contributor
  • 2355 Views
  • 4 replies
  • 1 kudos

how to disable delete option in data UI?

Hi, I need to remove the delete option from the data menu, how could I do it?

delete data option UI
Data Engineering
Reply
  • 2355 Views
  • 4 replies
  • 1 kudos
View Replies
Latest Reply
jose_gonzalez
Databricks Employee
  • 1 kudos

Hi @Andres Acuña​,Just a friendly follow-up. Did any of the responses help you to resolve your question? if it did, please mark it as best. Otherwise, please let us know if you still need help.

  • 1 kudos
3 More Replies
Mohan_Potla
by • New Contributor
  • 1696 Views
  • 2 replies
  • 0 kudos

Link to "Lakehouse Fundamentals Accreditation (V2)" in Databricks Academy

Please share link to take Lakehouse Fundamentals Accreditation (V2) in Databricks Academy.

Data Engineering
Reply
  • 1696 Views
  • 2 replies
  • 0 kudos
View Replies
Latest Reply
jose_gonzalez
Databricks Employee
  • 0 kudos

Adding @Vidula Khanna​ for visibility.

  • 0 kudos
1 More Replies
riccostamendes
by • New Contributor II
  • 2130 Views
  • 3 replies
  • 0 kudos

Reading a adls gen 2 csv file from db using standard pandas

i have mounted the adls container where the data are, but I cannot read the files with pandas ('pd.read_csv') even though I have attached the prefix '/dbfs/' to the path . Instead if I use 'spark.read.csv', I have no problems.Anyone knows why this i...

Data Engineering
Reply
  • 2130 Views
  • 3 replies
  • 0 kudos
View Replies
Latest Reply
Debayan
Databricks Employee
  • 0 kudos

Hi, When you are not able to read the files, what is the error you get?

  • 0 kudos
2 More Replies
Hubert-Dudek
by • Esteemed Contributor III
  • 1464 Views
  • 2 replies
  • 6 kudos

One great advantage of using the dDatabricks VSCode extension is that it doesn't require a personal access token. All you need to do is sign in to...

One great advantage of using the dDatabricks VSCode extension is that it doesn't require a personal access token. All you need to do is sign in to your Azure account, and you're good to go!

Untitled
Data Engineering
Reply
  • 1464 Views
  • 2 replies
  • 6 kudos
View Replies
Latest Reply
jose_gonzalez
Databricks Employee
  • 6 kudos

Indeed, thank you for sharing this!!!

  • 6 kudos
1 More Replies
sm1
by • New Contributor III
  • 2907 Views
  • 5 replies
  • 2 kudos

Can't run Display command. Any idea why I got this error message? How can I fix this? The nodes are at 28gb ram and memory usage : 162.9+mb so it shouldn't be memory issue :(. Please help! Thank you in advanced!

Capture
Data Engineering
Reply
  • 2907 Views
  • 5 replies
  • 2 kudos
View Replies
Latest Reply
Anonymous
Not applicable
  • 2 kudos

Hi @Suky Muliadikara​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.T...

  • 2 kudos
4 More Replies
jneira
by • New Contributor III
  • 3551 Views
  • 1 replies
  • 1 kudos

Setting database/catalog section of databricks jdbc driver url does not make it default

Hi, first of all thanks for the jdbc driver software.I have been used it with almost no problems but today i tried to set a a a database by default other than hive_metastore. I have followed the instructions and the example from the pdf jdbc driver g...

Data Engineering
Reply
  • 3551 Views
  • 1 replies
  • 1 kudos
View Replies
Latest Reply
sanjevraj
New Contributor III
  • 1 kudos

For anyone looking for an answer I had to add ConnCatalog=<catalog-name> in the URL.It is in the documentation https://docs.databricks.com/integrations/jdbc-odbc-bi.html#databricks-jdbc under JDBC driver capabilities.

  • 1 kudos
shan_chandra
by • Databricks Employee
  • 2528 Views
  • 1 replies
  • 1 kudos

Resolved! Adding spark_conf tag on Jobs API

using Jobs API, when we create a new job to run on an interactive cluster, can we add spark_conf tag and specify spark config tuning parameters?

Data Engineering
Reply
  • 2528 Views
  • 1 replies
  • 1 kudos
View Replies
Latest Reply
shan_chandra
Databricks Employee
  • 1 kudos

 spark_conf needs to be set prior to the start of the cluster or have to restart the existing cluster. Hence, the spark_conf tag is available only on the job_cluster. you may have to set the configs manually on the interactive cluster prior to using ...

  • 1 kudos
Hubert-Dudek
by • Esteemed Contributor III
  • 1757 Views
  • 3 replies
  • 5 kudos

Databricks added variable explorer to notebooks. It is beneficial when working on multiple cells and doesn&#39;t run them all. For example, we can see...

Databricks added variable explorer to notebooks. It is beneficial when working on multiple cells and doesn't run them all. For example, we can see variables that were initiated but from deleted cells (as that variables are still in cluster memory).

variables
Data Engineering
Reply
  • 1757 Views
  • 3 replies
  • 5 kudos
View Replies
Latest Reply
jose_gonzalez
Databricks Employee
  • 5 kudos

Great post, thank you for sharing

  • 5 kudos
2 More Replies

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group
Labels