cancel
Showing results for 
Search instead for 
Did you mean: 
Administration & Architecture
Explore discussions on Databricks administration, deployment strategies, and architectural best practices. Connect with administrators and architects to optimize your Databricks environment for performance, scalability, and security.
cancel
Showing results for 
Search instead for 
Did you mean: 
Data + AI Summit 2024 - Data Lakehouse Architecture


Forum Posts

Meghana_Vasavad
by New Contributor III
  • 475 Views
  • 1 replies
  • 0 kudos

Assistance Required for Enabling Unity Catalog in Databricks Workspace

Hi,I hope this message finds you well.I am reaching out regarding a concern with Databricks Administrator privileges. I have an Azure subscription and I use Azure Databricks for my tutorials, but I currently do not have Global Administrator access, w...

  • 475 Views
  • 1 replies
  • 0 kudos
Latest Reply
Walter_C
Databricks Employee
  • 0 kudos

Hello, each account should have at least one account administrator that is the one that has the permissions on the managed account and that can give you this access if applicable or assist you with the set up of the UC, you might need to ask internal...

  • 0 kudos
wsunwall
by New Contributor II
  • 678 Views
  • 1 replies
  • 1 kudos

Disabling Default Personal Compute Policy in Terraform (AWS)

I'm looking to disable the Personal Compute Policy in Terraform, however I don't see an option to do this. According to this documentation here https://docs.databricks.com/en/admin/clusters/personal-compute.html#customize-the-personal-compute-policy ...

  • 678 Views
  • 1 replies
  • 1 kudos
Latest Reply
mggl
New Contributor II
  • 1 kudos

I have the same issue in terraform-provider-google.I found how to do it from Admin UI (UI -> Settings -> Features Enablement -> disable Personal compute). At the same time I didn't find how to have the same behavior from Terraform.

  • 1 kudos
Edyta
by New Contributor II
  • 8722 Views
  • 1 replies
  • 0 kudos

Resolved! Delete Databricks account

Hi everyone, as in the topic, I would like to delete unnnecesarily created account. I have found outdated solutions (e.g. https://community.databricks.com/t5/data-engineering/how-to-delete-databricks-account/m-p/6323#M2501), but they do not work anym...

  • 8722 Views
  • 1 replies
  • 0 kudos
Latest Reply
szymon_dybczak
Contributor III
  • 0 kudos

Hi @Edyta ,FOR AWS:Manage your subscription | Databricks on AWS" Before you delete a Databricks account, you must first cancel your Databricks subscription and delete all Unity Catalog metastores in the account. After you delete all metastores associ...

  • 0 kudos
AmandaOhare
by New Contributor II
  • 1905 Views
  • 3 replies
  • 2 kudos

Resolved! Message queue to directly trigger job

Hi All,I'm very new to Databricks and trying to understand my use case a bit better.I have a databricks script / job that I want to be reactive to events outside of my databricks environment. A best case scenario would be if my script / job could aut...

  • 1905 Views
  • 3 replies
  • 2 kudos
Latest Reply
szymon_dybczak
Contributor III
  • 2 kudos

Hi @AmandaOhare ,You can use AWS Lambda to achieve that. You can setup queue trigger that will activate AWS lambda function. In that function you can call datbricks rest API that will launch workflow/job.

  • 2 kudos
2 More Replies
leo-machado
by New Contributor II
  • 699 Views
  • 1 replies
  • 3 kudos

Refresh permission on Lakeview Dashboard

Hi folks!I'm sure I'm not the only one, but our users have the tendency to click the big Refresh button on all dashboards every time they open them.Using resources efficiently is something I value deeply, so our team came up with a schedule policy - ...

image (2).png
  • 699 Views
  • 1 replies
  • 3 kudos
Latest Reply
holly
Databricks Employee
  • 3 kudos

Hi @leo-machado this is a very reasonable request. It's not feasible right now, but I'll pass it on to the product team as I'm sure you're not the only one. 

  • 3 kudos
pvilasra
by New Contributor II
  • 353 Views
  • 1 replies
  • 0 kudos

Unable to access databricls workspaces

Hi Team, Previously we are able to access databricks workspaces using username and password but now its asking for token, But we dont have access to mail account with username that we are using while login. Could you please assist on the issue how ca...

  • 353 Views
  • 1 replies
  • 0 kudos
Latest Reply
pvilasra
New Contributor II
  • 0 kudos

Anyone can help on this ?

  • 0 kudos
ma10
by New Contributor
  • 240 Views
  • 0 replies
  • 0 kudos

Issue with updating email with SCIM Provisioning

Hi all,For our set-up we have configured SCIM provisioning using Entra ID, group assignment on Azure is dealt with by IdentityIQ Sailpoint, and have enabled SSO for Databricks. It has/is working fine apart from one scenario. The original email assign...

  • 240 Views
  • 0 replies
  • 0 kudos
soumiknow
by New Contributor III
  • 238 Views
  • 0 replies
  • 0 kudos

How to add 'additionallyAllowedTenants' in Databricks config or PySpark config?

I have a multi-tenant Azure app. I am using this app's credentials to read ADLS container files from Databricks cluster using PySpark dataframe.I need to set this 'additionallyAllowedTenants' flag value to '*' or a specific tenant_id of the multi-ten...

  • 238 Views
  • 0 replies
  • 0 kudos
ADuma
by New Contributor III
  • 946 Views
  • 1 replies
  • 0 kudos

Resolved! Using Databricks Connect with PyCharm behind Proxy

I am using Databricks connect with PyCharm in order to locally develop my pipelines before deploying to Databricks. I set up authentication for databricks-connect using the following guide https://learn.microsoft.com/en-us/azure/databricks/dev-tools/...

  • 946 Views
  • 1 replies
  • 0 kudos
Latest Reply
ADuma
New Contributor III
  • 0 kudos

I managed to solve the problem by including my login and password in the HTTPS_PROXY environment variable.I set it to something like this HTTPS_PROXY = "http://username:password@proxy.company.com:port" and databricks-connect can create spark sessions...

  • 0 kudos
joe_zhou
by New Contributor III
  • 480 Views
  • 3 replies
  • 1 kudos

Can't login to new AWS databricks workspace

Just get our first AWS Databricks workspace created by following the instructions: https://docs.databricks.com/en/admin/workspace/create-uc-workspace.htmladopted the customer-managed VPC and private link for the setup. Upon reviewing the network conf...

joe_zhou_1-1723024460015.png joe_zhou_0-1723024191031.png
  • 480 Views
  • 3 replies
  • 1 kudos
Latest Reply
joe_zhou
New Contributor III
  • 1 kudos

the problem resolved after networking team currently set the DNS to point to the endpoint. We can login through the workspace link now. 

  • 1 kudos
2 More Replies
agarg
by New Contributor II
  • 1512 Views
  • 2 replies
  • 0 kudos

Databricks REST API to fetch mount points

Is there a way to fetch workspace  mount points ( mount infos) through REST API or SQL-query  ? ( similar to the python API "display(dbutils.fs.mounts())" )  I couldn't find any REST API for the mounts in the official databricks API  documentation ( ...

  • 1512 Views
  • 2 replies
  • 0 kudos
Latest Reply
pradeep1
New Contributor II
  • 0 kudos

How to create mount points using aws Databricks with shared cluster?

  • 0 kudos
1 More Replies
pv12
by New Contributor II
  • 990 Views
  • 4 replies
  • 3 kudos

Resolved! Does REST API usage is chargeable ?

If we use REST API to perform some operation on the Databricks cluster- like running some SQL Statements, will that be chargeable, will it costs us. 

  • 990 Views
  • 4 replies
  • 3 kudos
Latest Reply
szymon_dybczak
Contributor III
  • 3 kudos

Yep, to sum up - anything that requires compute power and makes the cluster to spin up will be charged.

  • 3 kudos
3 More Replies
AlbertWang
by Contributor III
  • 950 Views
  • 3 replies
  • 1 kudos

Resolved! Confusion about SCC, NAT, and Internet connection (Azure)

Hi all,I created an Azure Databricks workspace with the below featuresDeployed in my Azure virtual network (VNet injection)Enabled Secure cluster connectivityEnabled the Back-end Private LinkI did not create and use a NAT. I found that I still can ac...

  • 950 Views
  • 3 replies
  • 1 kudos
Latest Reply
szymon_dybczak
Contributor III
  • 1 kudos

Hi @AlbertWang ,If you choose SCC option, there shouldn't be NAT gateway created. You can check that yourself if you go the managed resource group that is created when you deploying databricks workspace.When you use SCC then if you would like to have...

  • 1 kudos
2 More Replies

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group
Labels