cancel
Turn on suggestions
Showing results for 
Search instead for 
Did you mean: 
Help
Administration & Architecture
Explore discussions on Databricks administration, deployment strategies, and architectural best practices. Connect with administrators and architects to optimize your Databricks environment for performance, scalability, and security.
cancel
Turn on suggestions
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Start a conversation
jkdatabricks
by • New Contributor
  • 888 Views
  • 1 replies
  • 0 kudos

Error: PERMISSION_DENIED: AWS IAM role does

Hello, We are trying to setup a new workspace. However we are getting following error.  Workspace failed to launch.Error: PERMISSION_DENIED: AWS IAM role does not have READ permissions on url s3://jk-databricks-prods3/unity-catalog/742920957025975.Pl...

Administration & Architecture
Reply
  • 888 Views
  • 1 replies
  • 0 kudos
View Replies
Latest Reply
caldempseyai
New Contributor II
  • 0 kudos

Hey! I'm experiencing this with the latest Terraform release. Try 1.51.0 if you are deploying via TF, downgrading fixed this for me.

  • 0 kudos
dpc
by • New Contributor III
  • 857 Views
  • 2 replies
  • 0 kudos

table deployment (DDL) from one catalog to another

HelloWe have a development, a test and a production environmentHow do you generally deploy DDL changes?So, alter a table in development and apply to test then productione.g.table1 has column1, column2, column3I add column4I now want to deploy this ch...

Administration & Architecture
Reply
  • 857 Views
  • 2 replies
  • 0 kudos
View Replies
Latest Reply
dpc
New Contributor III
  • 0 kudos

Thanks.I'll step through this solution and see if I can get it working

  • 0 kudos
1 More Replies
weilin0323
by • New Contributor III
  • 368 Views
  • 1 replies
  • 0 kudos

Testing and Issues Related to Admin Role Changes

Hello,I would like to ask a question regarding user permissions.Currently, all team members are admins. Recently, we plan to change the admin roles so that only I and another user, A, will be admins. The other members will retain general usage permis...

Administration & Architecture
Reply
  • 368 Views
  • 1 replies
  • 0 kudos
View Replies
Latest Reply
weilin0323
New Contributor III
  • 0 kudos

Hi @Kaniz_Fatma , Can you please help me to delete my post as I accidentally resubmitted the post.Thank you.

  • 0 kudos
cyborg-de
by • New Contributor
  • 837 Views
  • 0 replies
  • 0 kudos

DNS resolution across vnet

Hi, I have created a new databricks workspace in Azure with backend private link. Settings are Required NSG rules - No Azure Databricks RuleNSG rules for AAD and azfrontdoor were added as per documentation. Private endpoint with subresource  databric...

Administration & Architecture
Reply
  • 837 Views
  • 0 replies
  • 0 kudos
MosefJJunior3rd
by • New Contributor
  • 565 Views
  • 1 replies
  • 0 kudos

How to fully enable row-level concurrency on Databricks 14.1

Hey guys,I hope whoever's reading this is doing well.We're trying to enable row-level concurrency on Databricks 14.1. However, the documentation seems a bit contradictory as to whether this is possible, and whether the whole capability of row-level c...

Administration & Architecture
Reply
  • 565 Views
  • 1 replies
  • 0 kudos
View Replies
Latest Reply
gchandra
Databricks Employee
  • 0 kudos

Yes, your understanding is correct. As best practice, use 14.3 LTS DBR, as LTS has longer support compared to 6 months.

  • 0 kudos
JohnHarmer
by • New Contributor
  • 368 Views
  • 1 replies
  • 0 kudos

Unable to publish notebook

Hello, I receive an error each time I attempt to publish my notebook. the error message does not provide any additional information on why it is occurring. I need to publish this for a bootcamp assignment, so please let me know what I can do to resol...

Administration & Architecture
Reply
  • 368 Views
  • 1 replies
  • 0 kudos
View Replies
Latest Reply
gchandra
Databricks Employee
  • 0 kudos

Try it from a different browser, or try it in incognito mode.

  • 0 kudos
saravjeet
by • New Contributor III
  • 647 Views
  • 2 replies
  • 0 kudos

Allow non-admin users to view the driver logs from a Unity Catalog-enabled pipeline

We are trying to enable the option to allow log reading to the non-admin users in  the databricks wokspace but we are not able to see the obvious way to check them. The documentation is not showing after enabling the below property where to check the...

Administration & Architecture
Reply
  • 647 Views
  • 2 replies
  • 0 kudos
View Replies
Latest Reply
gchandra
Databricks Employee
  • 0 kudos

Do they see these links in the pipeline?

  • 0 kudos
1 More Replies
alexacas
by • New Contributor II
  • 1502 Views
  • 2 replies
  • 0 kudos

Resolved! Help with Databricks SQL Queries

Hi everyone,I’m relatively new to Databricks and trying to optimize some SQL queries for better performance. I’ve noticed that certain queries take longer to run than expected. Does anyone have tips or best practices for writing efficient SQL in Data...

Administration & Architecture
Reply
  • 1502 Views
  • 2 replies
  • 0 kudos
View Replies
Latest Reply
filipniziol
Esteemed Contributor
  • 0 kudos

Hi @alexacas ,The best thing is to share the queries and table structures But my general approach is:1. Use partitioning/zordering, or if you can upgrade runtime to 15.4, use liquid clustering, that is the new optimization technique.2. Make sure you ...

  • 0 kudos
1 More Replies
jmsimonramos1
by • New Contributor II
  • 1735 Views
  • 3 replies
  • 0 kudos

Grant permissions to groups on catalogs linked to the same metastore

Hi everyone!I am configuring several projects using Databricks, and I have a question regarding permission management in Unity Catalog. Here's the situation: Currently, I have two different Databricks resources in an Azure account, each with its resp...

Administration & Architecture
catalog
permissions
Unity Catalog
Reply
  • 1735 Views
  • 3 replies
  • 0 kudos
View Replies
Latest Reply
CharlesWoo
New Contributor II
  • 0 kudos

You could bind the catalog to specific workspaces, making them accessbile only from workspaces they are bound to.https://docs.databricks.com/en/catalogs/binding.htmlIn your example:if `my_catalog_2` is bound to `my_workspace_2` a user in `my_workspac...

  • 0 kudos
2 More Replies
Saf4Databricks
by • New Contributor III
  • 419 Views
  • 1 replies
  • 0 kudos

Databricks Free Trial - can I link it back from AWS to Azure?

I created a 14-day trial account on Databricks.com and linked it to my AWS. I have not used the trial yet and want to link it to Azure instead. Question: Can I do it, and if so, how?Remark: Please note that I have not used trial yet and the only rele...

Administration & Architecture
Reply
  • 419 Views
  • 1 replies
  • 0 kudos
View Replies
Latest Reply
gchandra
Databricks Employee
  • 0 kudos

The easy way will be to create a new account in portal.azure.com and launch Azure Databricks with Pay as you go.

  • 0 kudos
d_kailthya
by • New Contributor
  • 943 Views
  • 0 replies
  • 0 kudos

Implementing Databricks Persona in

Hi all,I am looking to implement the "persona" based access control across multiple workspaces for multiple user groups in Azure Databricks workspaces. Specifically,- I have a "DEV" workspace where the developer groups (Data Engineers and ML Engineer...

Administration & Architecture
Reply
  • 943 Views
  • 0 replies
  • 0 kudos
Junda
by • New Contributor III
  • 773 Views
  • 0 replies
  • 0 kudos

How to install private repository as package dependency in Databricks Workflow

I am a member of the development team in our company and we use Databricks as sort of like ETL tool. We utilize git integration for our program and run Workflow daily basis. Recently, we created another company internal private git repository and wan...

Administration & Architecture
Reply
  • 773 Views
  • 0 replies
  • 0 kudos
mkieszek
by • New Contributor II
  • 1148 Views
  • 4 replies
  • 1 kudos

set access control list for job cluster

Hello,currently, we are using an init script that calls the DBW API to add "can_attach_to" permissions for a specific group to initialize the job cluster. How can we set an access control list for job clusters? Is it possible to add it to a policy?

Administration & Architecture
Reply
  • 1148 Views
  • 4 replies
  • 1 kudos
View Replies
Latest Reply
szymon_dybczak
Esteemed Contributor III
  • 1 kudos

Hi @mkieszek ,You can use REST API to set access control list for job clusters. The endpoint is for doing that you can find below/api/2.0/permissions/clusters/{cluster_id}  (https://docs.databricks.com/api/workspace/clusters/setpermissions)And it's n...

  • 1 kudos
3 More Replies
funsjanssen
by • New Contributor III
  • 1766 Views
  • 0 replies
  • 7 kudos

Azure Databricks Multi Tenant Solution

Hello Everyone,For the past few months, we’ve been extensively exploring the use of Databricks as the core of our data warehousing product. We provide analytics dashboards to other organizations and are particularly interested in the Column-Level Sec...

Administration & Architecture
Reply
  • 1766 Views
  • 0 replies
  • 7 kudos
AlbertWang
by • Valued Contributor
  • 1228 Views
  • 5 replies
  • 2 kudos

Resolved! Databricks bundle custom variables feature does not work for workspace host

Hi all,I am not sure whether this is a bug. I followed the document.My databricks.yml file: bundle: name: test variables: DATABRICKS_HOST: description: The host of the Databricks workspace. DATABRICKS_JOB_RUNNING_SERVICE_PRINCIPAL_CLIENT_I...

Administration & Architecture
Reply
  • 1228 Views
  • 5 replies
  • 2 kudos
View Replies
Latest Reply
filipniziol
Esteemed Contributor
  • 2 kudos

Hi @AlbertWang ,The documentation states you cannot map workspace host: 

  • 2 kudos
4 More Replies

Join Us as a Local Community Builder!

Passionate about hosting events and connecting people? Help us grow a vibrant local community—sign up today to get started!

Sign Up Now
Top