cancel
Showing results for 
Search instead for 
Did you mean: 
Administration & Architecture
Explore discussions on Databricks administration, deployment strategies, and architectural best practices. Connect with administrators and architects to optimize your Databricks environment for performance, scalability, and security.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

heathwinning
by New Contributor II
  • 1721 Views
  • 1 replies
  • 0 kudos

Obtain access of Azure metastore storage account to configure Lifecycle management

I recently set up an Azure Databricks workspace with an automatically created metastore and metastore-level root storage within the metastore blob storage account. All the catalogs, schemas, and tables/volumes have been created without a specified or...

  • 1721 Views
  • 1 replies
  • 0 kudos
prateekgoyal
by New Contributor II
  • 3038 Views
  • 2 replies
  • 0 kudos

Databricks connection pool(ing) using python

I have a python application which I need to connect with databricks (Databricks SQL Connector). The process for that is quite simple -1. connect with dbr2. execute your query3. get the result4. close the connectionExample from - https://docs.databric...

  • 3038 Views
  • 2 replies
  • 0 kudos
Latest Reply
prateekgoyal
New Contributor II
  • 0 kudos

Hi @Retired_mod, thanks for your reply. I have 2 follow-up questions for you - When you say "consider managing connection lifetimes by closing and recreating them periodically" - the problem I face with this is when my API request comes in and I crea...

  • 0 kudos
1 More Replies
antonuzzo96
by New Contributor III
  • 4345 Views
  • 5 replies
  • 4 kudos

Resolved! UCX- installation error -

I'm getting the following error when trying to install UCX on  a specific workspace.I have already installed ucx from my machine on other workspace and works, now on this workspace i'm getting this error  ERROR [d.l.blueprint.parallel][installing_das...

  • 4345 Views
  • 5 replies
  • 4 kudos
Latest Reply
Cor
New Contributor III
  • 4 kudos

Issue was resolved in lsql v0.7.4, a ucx dependency: https://github.com/databrickslabs/lsql/releases/tag/v0.7.41. Upgrade the Databricks cli: https://github.com/databricks/cli/releases/2. Upgrade ucx: `databricks labs install ucx`

  • 4 kudos
4 More Replies
cleversuresh
by New Contributor III
  • 991 Views
  • 1 replies
  • 0 kudos

vocareum lab showing no workspace found.

Hi,I bought Generative AI Engineering with Databricks lab subscription by paying $75 just to get the hands on experience.The labs worked for one day and then they stop working. Please active the labs ASAP, I am preparing for the certification exam an...

  • 991 Views
  • 1 replies
  • 0 kudos
Latest Reply
Retired_mod
Esteemed Contributor III
  • 0 kudos

Hi @cleversuresh, Thank you for sharing your concern with us!   To expedite your request, please list your concerns on our ticketing portal. Our support staff would be able to act faster on the resolution (our standard resolution time is 24-48 hours)...

  • 0 kudos
StephanieAlba
by Databricks Employee
  • 2749 Views
  • 5 replies
  • 2 kudos

Promoting assets from Dev workspace to Prod

What is the best way to promote Databricks objects from one workspace to another? For example, DABs are great for code and models. However, SQL alerts do not seem to fall into DABs. The API allows for retrieving queries and creating them via API in t...

  • 2749 Views
  • 5 replies
  • 2 kudos
Latest Reply
NandiniN
Databricks Employee
  • 2 kudos

https://docs.databricks.com/api/workspace/queries https://docs.databricks.com/en/sql/dbsql-api-latest.html#changes-to-the-alerts-api The API path is now api/2.0/sql/alerts, replacing the legacy path of /api/2.0/preview/sql/alerts. https://docs.databr...

  • 2 kudos
4 More Replies
millie_dbrx_use
by New Contributor
  • 1108 Views
  • 1 replies
  • 1 kudos

System Tables

Hi,I've recently enabled the billing schema in system tables, but still not able to view it in catalog. I am both an account admin and workspace admin, as well as an owner of the workspace in Azure. A colleague (who used to be Azure AD admin but no l...

  • 1108 Views
  • 1 replies
  • 1 kudos
ag2all
by New Contributor
  • 1271 Views
  • 1 replies
  • 1 kudos

Databricks + Apache Iceberg = advantageous or wasted effort due to duplicate functionality ?

Trying to design a Lakehouse. Spark is at the base layer. Now wondering if adding Apache Iceberg sitting below Spark will be of help, or, not ? Preferring Iceberg for its auto indexing, ACID query facilities over big hetergenous datasets. Wonder if i...

  • 1271 Views
  • 1 replies
  • 1 kudos
Latest Reply
holly
Databricks Employee
  • 1 kudos

Hello, if you're planning on building your own open source stack of spark+iceberg, it can be a good choice.  If you're on Databricks, however, you're going to miss out a *lot* on delta features that are baked into the platform. Specifically compute +...

  • 1 kudos
data_bricklayer
by New Contributor III
  • 4655 Views
  • 4 replies
  • 3 kudos

Resolved! [VNET injection] Container and container subnet

Hi,I was researching everywhere and could not find the answer. I understand that when workspace is created, it has 2 subnets, host and container. The VM, which runs the Databricks container, is in host subnet, which logically means the container is a...

  • 4655 Views
  • 4 replies
  • 3 kudos
Latest Reply
szymon_dybczak
Esteemed Contributor III
  • 3 kudos

Hi @data_bricklayer ,Public Subnet (host): The public subnet is typically used for resources that need to communicate with the internet or other Azure services. In Azure Databricks, this subnet is used for driver nodes of the clusters that require ou...

  • 3 kudos
3 More Replies
PetrSindelar
by New Contributor II
  • 5634 Views
  • 3 replies
  • 2 kudos

Connecting external location from a different tenant in Azure

Hi,we have a setup with 2 different Azure tenants. In tenant A we have a storage account that we want to connect as an external location to a databricks workspace in the tenant B. For that we have established a private endpoint from the storage accou...

  • 5634 Views
  • 3 replies
  • 2 kudos
Latest Reply
Behwar
New Contributor III
  • 2 kudos

It would be superb to connect between two tenants with Azure Databricks Access Connector

  • 2 kudos
2 More Replies
TheManOfSteele
by New Contributor III
  • 4168 Views
  • 1 replies
  • 0 kudos

Resolved! Prevent service principal UUID from appearing on job name

Hello!I am using service principal id to authenticate my databricks bundle. But when the job runs, this id is automatically appended to both the name and tags column on the jobs run page. In my databricks.yml file I have name: "[${var.environment}]" ...

  • 4168 Views
  • 1 replies
  • 0 kudos
Latest Reply
breaka
New Contributor III
  • 0 kudos

Hi!Sounds like the "development" mode. DAB will automatically prefix your jobname with <env> <user name> if you set "mode" to "development" in the databricks.yml file. The name lookup for service principals apparently doesn't work nicely and you get ...

  • 0 kudos
Maria_S
by New Contributor III
  • 2334 Views
  • 4 replies
  • 5 kudos

Why all workspace users can see my user folder

Hi, I am a Databricks account admin user with admin access to our workspace. My user folder for some reason is visible to all workspace users. I have checked permissions settings where possible and cannot see anything that would indicate fully shared...

Capture1.PNG Capture2.PNG
  • 2334 Views
  • 4 replies
  • 5 kudos
Latest Reply
Aviral-Bhardwaj
Esteemed Contributor III
  • 5 kudos

workspace is visible for all , you have to make changes in Admin console ,you will find this feature there to disable it 

  • 5 kudos
3 More Replies
TheManOfSteele
by New Contributor III
  • 2088 Views
  • 1 replies
  • 1 kudos

Resolved! How to Pass Azure variable to databricks.yml file

Hello I would like to find a way to pass a variable from my Azure variables to my databricks yml file.    For example I would like to pass the variable BUNDLE_TARGET to the location in this databricks.yml fileIs there a way to do something like this?...

TheManOfSteele_0-1722601812560.png TheManOfSteele_1-1722601836664.png TheManOfSteele_3-1722601999059.png
  • 2088 Views
  • 1 replies
  • 1 kudos
Latest Reply
szymon_dybczak
Esteemed Contributor III
  • 1 kudos

Hi @TheManOfSteele ,Here are the examples of how to achieve that. I think the simplest way would be to set environment variables.azure devops - How can I pass parameters to databricks.yml in Databricks Asset Bundles? - Stack OverflowDatabricks Asset ...

  • 1 kudos
TheManOfSteele
by New Contributor III
  • 3589 Views
  • 0 replies
  • 0 kudos

Azure Pipeline Release Bundle Validate Failure

Hello,I am trying to create a CI/CD pipeline. After the build pipeline, I am trying to create a release to databricks that runs a notebook. I am trying to run this as a service principal. During the bundle validate step I am getting this error. "Erro...

TheManOfSteele_0-1722516587122.png
  • 3589 Views
  • 0 replies
  • 0 kudos
gor
by New Contributor III
  • 1933 Views
  • 3 replies
  • 0 kudos

Resolved! Workspace selector not working

On the top-right of the Databricks GUI is a selector to select workspaces from.Since yesterday morning (approximately the same time the Microsoft outage happened) that selector stopped working.  Instead of a dropdown of Workspaces, we only get a spin...

  • 1933 Views
  • 3 replies
  • 0 kudos
Latest Reply
NandiniN
Databricks Employee
  • 0 kudos

Glad to know the issue has stopped occurring, but in case it re-occurs in future, we could collect some backend logs, to understand what is causing the slowness, when the issue is and resolve the service. I would request you to involve the azure team...

  • 0 kudos
2 More Replies