cancel
Showing results for 
Search instead for 
Did you mean: 
Administration & Architecture
Explore discussions on Databricks administration, deployment strategies, and architectural best practices. Connect with administrators and architects to optimize your Databricks environment for performance, scalability, and security.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

l0ginp
by New Contributor III
  • 120 Views
  • 2 replies
  • 0 kudos

Resolved! I have questions about "Premium Automated Serverless Compute - Promo DBU."

"Premium Automated Serverless Compute - Promo DBU" expenses arise from what, how can I disable it, and why are the costs so high? In the picture, I am using AzureThank you in advance for the advice  

l0ginp_2-1717951633113.png
  • 120 Views
  • 2 replies
  • 0 kudos
Latest Reply
l0ginp
New Contributor III
  • 0 kudos

Hello everyone!Currently, I found that over 90% of the cost from "Premium Automated Serverless Compute - Promo DBU" is due to Databricks Materialized View working with Job + SQL Warehouse Serverless. I have now resolved the issue by discontinuing the...

  • 0 kudos
1 More Replies
adb-rm
by New Contributor II
  • 149 Views
  • 2 replies
  • 0 kudos

Unity catalog and metadata to backup site

Hi ,I am looking for Databricks DR(Disaster recovery)  site creation. My  primary setup is in west us and my primary data bricks created in primary site with unity catalog enable. with same meta data i want bring up the Databricks in my secondary sid...

  • 149 Views
  • 2 replies
  • 0 kudos
Latest Reply
Walter_C
Valued Contributor III
  • 0 kudos

As of now UC does not have DR setting, this is currently in our prioritized roadmap but no ETA is currently available. Some customers host an external metastore and replicate it across regions to achieve this.

  • 0 kudos
1 More Replies
mh_db
by New Contributor III
  • 36 Views
  • 0 replies
  • 0 kudos

link databrick to AWS GOV cloud

Does anyone know how to link databricks to AWS GOV cloud? I subscribed to databricks and created databricks account from the link in Marketplace. When I go to databricks account console the URL doesn't take me to the URL reference in the documentatio...

  • 36 Views
  • 0 replies
  • 0 kudos
Carsten03
by New Contributor III
  • 5154 Views
  • 6 replies
  • 1 kudos

Run workflow using git integration with service principal

Hi,I want to run a dbt workflow task and would like to use the git integration for that. Using my personal user I am able to do so but I am running my workflows using a service principal.I added git credentials and the repository using terraform. I a...

  • 5154 Views
  • 6 replies
  • 1 kudos
Latest Reply
justOnePost
New Contributor
  • 1 kudos

Hi @Carsten03 , @MiroFuoli ,I had a very similar issue and finally got it to work. Though this pretty much depends on your setup. I am using azure databricks and bitbucket, if you have a similar setup the following hopefully works for you. Create an ...

  • 1 kudos
5 More Replies
mh_db
by New Contributor III
  • 64 Views
  • 0 replies
  • 0 kudos

Databricks AWS ACLs requirement

I'm going through the documentation in databricks for using existing VPC requirements. For the section that covers ACLs, what's the need to have this inbound rule: " ALLOW ALL from Source 0.0.0.0/0. This rule must be prioritized."https://docs.databri...

Administration & Architecture
Architecture
AWS
VPC
  • 64 Views
  • 0 replies
  • 0 kudos
Avvar2022
by Contributor
  • 1704 Views
  • 2 replies
  • 0 kudos

Is there a setting which restricts users from Creating Job and Pipeline?

as far i know currently ((as of 03-25-2024) databricks don't any workspace admin settings option to restrict users from creating a workflow/job or delta pipelines. Here is the use case for it Example: you have 3 tier landscape Dev, Qa and Prod.It is ...

Administration & Architecture
administration
jobs
pipelines
  • 1704 Views
  • 2 replies
  • 0 kudos
Latest Reply
Avvar2022
Contributor
  • 0 kudos

@Kaniz - How can I initiate a product enhancement request?

  • 0 kudos
1 More Replies
Etyr
by Contributor
  • 553 Views
  • 3 replies
  • 0 kudos

Unity Catalog read issue

Hello,Our company is POCing the Unity Catalog with Azure as provider.We have 2 subscriptions that contains 1 databricks each and 1 ADLS GEN2 each.Initially we have the default `hive_metastore` connected to the ADLS GEN2. I've created a secret scope a...

Etyr_0-1717660060863.png Etyr_1-1717660177530.png Etyr_2-1717663031618.png
  • 553 Views
  • 3 replies
  • 0 kudos
Latest Reply
Etyr
Contributor
  • 0 kudos

Hello,The UC catalog container is sitting in Sub B.Basically we have a Spoke and Hub configuration for each Subscriptions. Each Subscriptions can access to any ressource inside it own subscription with some PE. But to access other Subscription ressou...

  • 0 kudos
2 More Replies
aghiya
by New Contributor
  • 182 Views
  • 1 replies
  • 0 kudos

Using JARs from Google Cloud Artifact Registry in Databricks for Job Execution

We have our CI/CD pipelines set up in Google Cloud using Cloud Build, and we are publishing our artifacts to a private repository in Google Cloud's Artifact Registry. I want to use these JAR files to create and run jobs in Databricks.However, when I ...

  • 182 Views
  • 1 replies
  • 0 kudos
Latest Reply
brijeshgoud1
New Contributor II
  • 0 kudos

To integrate your CI/CD pipeline with Databricks, you have a couple of options:1. Using Artifact Registry as an intermediary: Currently, Databricks does not directly support Google Artifact Registry URLs. However, you can use an intermediate storage ...

  • 0 kudos
tonypiazza
by New Contributor
  • 154 Views
  • 1 replies
  • 0 kudos

Enable git credential for entire Databricks account

Hello,I have a question regarding git authentication in the context of a Databricks job. I know that a GitHub Personal Access Token can be generated for a user using GitHub App authorization flow. My team is currently configuring Terraform templates ...

  • 154 Views
  • 1 replies
  • 0 kudos
Latest Reply
Yeshwanth
Honored Contributor
  • 0 kudos

@tonypiazza good day! To authorize access to all private repositories within a GitHub organization at the account level, without configuring git credentials on a per-user basis, follow these steps: 1. Use a GitHub Machine User:- Create a GitHub machi...

  • 0 kudos
Conlyn
by New Contributor II
  • 438 Views
  • 2 replies
  • 0 kudos

Resolved! Terraform databricks_grants errors on external_location

We are using terraform to setup Unity Catalog external locations and when using databricks_grants to set permissions on the external locations it throws the following error:  Error: cannot create grants: permissions for external_location-test_locatio...

  • 438 Views
  • 2 replies
  • 0 kudos
Latest Reply
Conlyn
New Contributor II
  • 0 kudos

I figured out my issue... The principal name is case sensitive and if the input value doesn't match the case of the email address or Group Name in the workspace/account it throws that ambiguous error.  

  • 0 kudos
1 More Replies
Join 100K+ Data Experts: Register Now & Grow with Us!

Excited to expand your horizons with us? Click here to Register and begin your journey to success!

Already a member? Login and join your local regional user group! If there isn’t one near you, fill out this form and we’ll create one for you to join!

Labels