cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

JordanYaker
by Contributor
  • 2625 Views
  • 1 replies
  • 1 kudos

What is the maximum number of workspaces per account using Databricks on AWS?

I've been looking through the documentation and I swear this used to be listed somewhere, but for the life of me I can't find it anymore.

  • 2625 Views
  • 1 replies
  • 1 kudos
Latest Reply
JordanYaker
Contributor
  • 1 kudos

Thanks @Kaniz Fatma​ 

  • 1 kudos
kkawka1
by New Contributor III
  • 13562 Views
  • 7 replies
  • 10 kudos

Resolved! Removing files saved in the root FileStore

We have just started working with databricks in one of my university modules, and the lecturers gave us a set of commands to practice saving data in the FileStore. One of the commands was the following:dbutils .fs.cp("/ databricks - datasets / weathh...

  • 13562 Views
  • 7 replies
  • 10 kudos
Latest Reply
-werners-
Esteemed Contributor III
  • 10 kudos

you can delete files using the data explorer in Databricks web UI.another option is to use %fs or %sh in a notebook.

  • 10 kudos
6 More Replies
Shanthala
by New Contributor III
  • 1796 Views
  • 1 replies
  • 3 kudos

Workspace usage for the partners

We have 11 people working on the Data Engineering Associate certification using Data Engineering with Databricks V3.  We just got done with the Foundation one and start the Engineering journey. We are Registered partners and Data Engineering with Dat...

  • 1796 Views
  • 1 replies
  • 3 kudos
Latest Reply
youssefmrini
Databricks Employee
  • 3 kudos

Hell Shanthala, you can send an email to partnerops@databricks.com who then provide information how to set this up

  • 3 kudos
killjoy
by New Contributor III
  • 9549 Views
  • 7 replies
  • 0 kudos

Resolved! Pipeline failed while calling Databricks Notebook - Cluster Terminated

Hello,We have an Azure Data Factory pipeline running during the night, and one of the activities calls a Databricks Notebook with dynamic DatabricksInstancePoolId, ClusterVersion and Workers. Yesterday, it failed with with the following error:Cluster...

  • 9549 Views
  • 7 replies
  • 0 kudos
Latest Reply
jose_gonzalez
Databricks Employee
  • 0 kudos

Hi @Rita Fernandes​,What are you trying to install in your init script? only the ODBC driver or some other libraries/dependencies?

  • 0 kudos
6 More Replies
grazie
by Contributor
  • 3412 Views
  • 2 replies
  • 0 kudos

Resolved! slack notification (webhook) failing

POST to the slack webhook from local http client works_"Test" action on "System Notifcations" page in Databricks gives response 400 bad request

  • 3412 Views
  • 2 replies
  • 0 kudos
Latest Reply
grazie
Contributor
  • 0 kudos

All in all, I got confused by a misconfiguration.A notification was setup as a webhook notificaton instead of a slack notification by mistake in a particular workspace, which caused the confusion. So, "problem in chair, not computer". If anything, it...

  • 0 kudos
1 More Replies
DianGermishuiz1
by New Contributor III
  • 10426 Views
  • 7 replies
  • 4 kudos

Resolved! Unsupported cloud file system schema 'wasbs' error in custom catalog, but not in hive_metastore

When trying to run a create table USING CSV that pulls data from an azure blob storage path in a custom catalog I created, I get an error stating Unsupported cloud file system schema 'wasbs'.However, when I run this code in the hive_metastore catalog...

wasbs cloud path unsupported error
  • 10426 Views
  • 7 replies
  • 4 kudos
Latest Reply
youssefmrini
Databricks Employee
  • 4 kudos

WASB Has been deprecated.

  • 4 kudos
6 More Replies
Databrickguy
by New Contributor II
  • 4166 Views
  • 2 replies
  • 2 kudos

How to check/list the tables which has CDF enabled?

How to list all the tables which has CDF enabled?I can review a table to find out if CDF is enable with below code.SHOW TBLPROPERTIES tableA(delta.enableChangeDataFeed)The return key ...

  • 4166 Views
  • 2 replies
  • 2 kudos
Latest Reply
jose_gonzalez
Databricks Employee
  • 2 kudos

Hhi @Tim zhang​,Just a friendly follow-up. Did any of the responses help you to resolve your question? if it did, please mark it as best. Otherwise, please let us know if you still need help.

  • 2 kudos
1 More Replies
amitca71
by Contributor II
  • 4269 Views
  • 6 replies
  • 0 kudos

Customer-managed VPC with empty default security group

Hi,I use self managed VPC. when i use security group the jobs clusters are being created.if i make the default security group empty and create customized security group (required by soc2 to have empty default), with same definition of ingress (allow...

  • 4269 Views
  • 6 replies
  • 0 kudos
Latest Reply
jose_gonzalez
Databricks Employee
  • 0 kudos

Hi @Amit Cahanovich​,Just a friendly follow-up. Did any of the responses help you to resolve your question? if it did, please mark it as best. Otherwise, please let us know if you still need help.

  • 0 kudos
5 More Replies
eacuna
by New Contributor
  • 3219 Views
  • 4 replies
  • 1 kudos

how to disable delete option in data UI?

Hi, I need to remove the delete option from the data menu, how could I do it?

delete data option UI
  • 3219 Views
  • 4 replies
  • 1 kudos
Latest Reply
jose_gonzalez
Databricks Employee
  • 1 kudos

Hi @Andres Acuña​,Just a friendly follow-up. Did any of the responses help you to resolve your question? if it did, please mark it as best. Otherwise, please let us know if you still need help.

  • 1 kudos
3 More Replies
Mohan_Potla
by New Contributor
  • 2444 Views
  • 2 replies
  • 0 kudos

Link to "Lakehouse Fundamentals Accreditation (V2)" in Databricks Academy

Please share link to take Lakehouse Fundamentals Accreditation (V2) in Databricks Academy.

  • 2444 Views
  • 2 replies
  • 0 kudos
Latest Reply
jose_gonzalez
Databricks Employee
  • 0 kudos

Adding @Vidula Khanna​ for visibility.

  • 0 kudos
1 More Replies
riccostamendes
by New Contributor II
  • 2880 Views
  • 3 replies
  • 0 kudos

Reading a adls gen 2 csv file from db using standard pandas

i have mounted the adls container where the data are, but I cannot read the files with pandas ('pd.read_csv') even though I have attached the prefix '/dbfs/' to the path . Instead if I use 'spark.read.csv', I have no problems.Anyone knows why this i...

  • 2880 Views
  • 3 replies
  • 0 kudos
Latest Reply
Debayan
Databricks Employee
  • 0 kudos

Hi, When you are not able to read the files, what is the error you get?

  • 0 kudos
2 More Replies
Hubert-Dudek
by Esteemed Contributor III
  • 1986 Views
  • 2 replies
  • 6 kudos

One great advantage of using the dDatabricks VSCode extension is that it doesn't require a personal access token. All you need to do is sign in to...

One great advantage of using the dDatabricks VSCode extension is that it doesn't require a personal access token. All you need to do is sign in to your Azure account, and you're good to go!

Untitled
  • 1986 Views
  • 2 replies
  • 6 kudos
Latest Reply
jose_gonzalez
Databricks Employee
  • 6 kudos

Indeed, thank you for sharing this!!!

  • 6 kudos
1 More Replies

Join Us as a Local Community Builder!

Passionate about hosting events and connecting people? Help us grow a vibrant local community—sign up today to get started!

Sign Up Now
Labels