cancel
Showing results for 
Search instead for 
Did you mean: 
Administration & Architecture
Explore discussions on Databricks administration, deployment strategies, and architectural best practices. Connect with administrators and architects to optimize your Databricks environment for performance, scalability, and security.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Xyguo
by New Contributor
  • 1800 Views
  • 1 replies
  • 1 kudos

Resolved! REST API workspace list content doesn't work with Queries

Hi, I'm trying to export the SQL Queries in certain folders in workspace, but the list content API callGET  /api/2.0/workspace/list  doesnt work with queries? how should I export only queries in a certain folder in workspace? Thank you very much  

Xyguo_0-1691335659836.png
  • 1800 Views
  • 1 replies
  • 1 kudos
Latest Reply
shan_chandra
Databricks Employee
  • 1 kudos

@Xyguo - Currently, exporting a SQL query file is not supported.  Kindly create an idea feature request by following the article listed - https://docs.databricks.com/en/resources/ideas.html#create-an-idea-in-the-ideas-portal to raise a feature reques...

  • 1 kudos
milicevica
by New Contributor
  • 8182 Views
  • 2 replies
  • 1 kudos

Resolved! Multiple orphan vms in managed resource group after starting and terminating my personal cluster

Hi today, i had problem with starting the cluster and therefore i did multiple times star and terminating. The problem is that this actions always started one new VM in the managed resource group but never turned it off. Therefore i ended up with mul...

  • 8182 Views
  • 2 replies
  • 1 kudos
Latest Reply
Hubert-Dudek
Esteemed Contributor III
  • 1 kudos

It always takes a few minutes, even 10, before the machine is terminated in Azure. It is better to set a pool of machines in data bricks and use them from there so we will not be keeping Zombie machines.

  • 1 kudos
1 More Replies
Hubert-Dudek
by Esteemed Contributor III
  • 2336 Views
  • 0 replies
  • 1 kudos

Databricks Enhances Job Monitoring with Duration Thresholds for Workflow

Databricks has introduced Duration Thresholds for workflows. This new addition allows users to set time limits for workflow execution, significantly improving monitoring of job performance. When a job exceeds the preset duration, the system triggers ...

ezgif-4-8ab9474b98.gif
  • 2336 Views
  • 0 replies
  • 1 kudos
vasu2
by New Contributor
  • 794 Views
  • 0 replies
  • 0 kudos

Multi bronze layers : multi client data

data coming in from multiple sources belonging to multiple customers.Should I create Single storage account name it bronze and then separate containers for each client's data to be put in.And then merge the data in the silver layer. What's the best p...

  • 794 Views
  • 0 replies
  • 0 kudos
niklas
by Contributor
  • 5827 Views
  • 1 replies
  • 0 kudos

Resolved! Can't set account admin using Terraform

I want to set the account admin for a service principal in order to create the Unity Catalog metastore. The Terraform code looks like this: data "databricks_service_principal" "application" { count = var.environment == "dev" ? 1 : 0 application_...

Administration & Architecture
access
account_admin
azure
role
Terraform
  • 5827 Views
  • 1 replies
  • 0 kudos
shawnbarrick
by New Contributor III
  • 4979 Views
  • 2 replies
  • 0 kudos

INVALID_PARAMETER_VALUE.LOCATION_OVERLAP: overlaps with managed storage error with S3 paths

We're trying to read from an S3 bucket using unity catalog and are selectively getting "INVALID_PARAMETER_VALUE.LOCATION_OVERLAP: overlaps with managed storage error"  errors within the same bucket.  This works: "dbutils.fs.ls("s3://BUCKETNAME/dev/he...

  • 4979 Views
  • 2 replies
  • 0 kudos
Latest Reply
Debayan
Databricks Employee
  • 0 kudos

Hi,Could you please elaborate on the issue here? Running the list command on a managed directory is not supported in Unity Catalog. Catalog/schema storage locations are reserved for managed storage.Please tag @Debayan  with your next comment which wi...

  • 0 kudos
1 More Replies
DianGermishuize
by New Contributor II
  • 1740 Views
  • 1 replies
  • 0 kudos

Bitbucket Repo Add Assistance

I am trying to add a Bitbucket cloud repo to my workspace, but keep getting a credentials invalid error. I made sure to set a username and generate an app password with repo read write permissions. What am i doing wrong? 

  • 1740 Views
  • 1 replies
  • 0 kudos
Latest Reply
Harrison_S
Databricks Employee
  • 0 kudos

Hello,Did you also set the password in User Settings > Git integration ? If so can you paste the full error?

  • 0 kudos
hattier
by New Contributor II
  • 3778 Views
  • 1 replies
  • 1 kudos

Resolved! OIDC account-level SSO and SAML workspace-level SSO

Hey folks! I am seeking clarity for this passage in documentation: "This means that SSO for your account and SSO for each workspace must be managed separately. You should configure SSO to the the same identity provider at the account and on your work...

  • 3778 Views
  • 1 replies
  • 1 kudos
Latest Reply
Prabakar
Databricks Employee
  • 1 kudos

If both are pointing to the same IDP then it should not be a problem.

  • 1 kudos
FrankBarten
by New Contributor II
  • 4013 Views
  • 5 replies
  • 0 kudos

How to hide workspace to which I don't have access anymore?

In my Account Console and workspace selector I'm seeing a list of 10 workspaces, but to half of them I don't have access anymore (as intended). How can I remove these workspaces which have become irrelevant for me from this view? 

FrankBarten_0-1688374156442.png
  • 4013 Views
  • 5 replies
  • 0 kudos
Latest Reply
Anonymous
Not applicable
  • 0 kudos

Hi @FrankBarten  Hope you are well. Just wanted to see if you were able to find an answer to your question and would you like to mark an answer as best? It would be really helpful for the other members too. Cheers!

  • 0 kudos
4 More Replies
Hubert-Dudek
by Esteemed Contributor III
  • 3003 Views
  • 1 replies
  • 5 kudos

Metrics are the lifeblood of every business. They help measure performance, track progress, and optimize decision-making. Unfortunately, however, most...

Metrics are the lifeblood of every business. They help measure performance, track progress, and optimize decision-making. Unfortunately, however, most projects' metrics are buried in code and scattered across different tools and platforms. This makes...

1684244491813
  • 3003 Views
  • 1 replies
  • 5 kudos
Latest Reply
Anonymous
Not applicable
  • 5 kudos

Thanks for sharing your knowledge with Community.

  • 5 kudos
Dharinip
by New Contributor III
  • 3406 Views
  • 1 replies
  • 0 kudos

Connection Error

I am getting this type of erro "com.databricks.spark.sqldw.SqlDWConnectorException: Exception encountered in Azure Synapse Analytics connector code" while loading the data into Synapse. I am able to load other tables with the same connection. Not sur...

  • 3406 Views
  • 1 replies
  • 0 kudos
Latest Reply
Anonymous
Not applicable
  • 0 kudos

The error message you provided indicates that an exception occurred in the Azure Synapse Analytics connector code while loading data into Synapse. This can happen due to various reasons, such as issues with the table structure, data type compatibilit...

  • 0 kudos
Lwin
by New Contributor
  • 964 Views
  • 1 replies
  • 0 kudos

Want to learn more about system tables

Where can I learn more about system tables? Looking for docs or usage stories! Thanks

  • 964 Views
  • 1 replies
  • 0 kudos
Latest Reply
Anonymous
Not applicable
  • 0 kudos

Please refer to this - https://docs.databricks.com/administration-guide/system-tables/index.html

  • 0 kudos
Ting
by New Contributor
  • 1541 Views
  • 1 replies
  • 0 kudos

Can I use Databricks as a Backend?

Hi, I am new to Databricks and I am exploring its CE. I have data science background and I am building an analytics webapp, instead of connecting to Supabase or other back-end services, is it possible for me to connect to Databricks directly? I could...

  • 1541 Views
  • 1 replies
  • 0 kudos
Latest Reply
Anonymous
Not applicable
  • 0 kudos

How about the Databricks REST API? DB offers a comprehensive REST API that allows you to programmatically interact with Databricks resources. You can use this API to submit jobs, run queries, manage clusters, and perform various other operations. 

  • 0 kudos
yubin-apollo
by New Contributor II
  • 914 Views
  • 1 replies
  • 0 kudos

Single Sign On with Duo - Redirect Error

Hi,Has anybody tried using Duo for SSO (Single Sign On)?We used OIDC on Duo, and getting this error when we test. Basically, after we get authenticated by Duo, the redirect page shows up like this:Any tips would be greatly appreciated. Thanks!

screencapture-accounts-cloud-databricks-login-2023-06-30-19_18_44.png
  • 914 Views
  • 1 replies
  • 0 kudos
Latest Reply
Anonymous
Not applicable
  • 0 kudos

Hi @yubin-apollo We are reaching out to inform you that Databricks Community has migrated to a new service provider. As a result, all users and published content are migrated on June 24, 2023.This change will enhance your overall Databricks Community...

  • 0 kudos