- 2621 Views
- 7 replies
- 0 kudos
unknown geo redundancy storage events (& costs) in azure databricks resource group
Hi All,I'm after some guidance on how to identify massive (100000%) spikes in bandwidth usage (and related costs) in the azure databricks provisioned/managed resource group storage account & stop themThese blips are adding 30-50% to our monthly costs...
- 2621 Views
- 7 replies
- 0 kudos
- 1694 Views
- 6 replies
- 0 kudos
Unity catalog meta store is created within undesired storage account
I came to know that our unity catalog meta store has been created in the default storage account of our databricks workspace and this storage account has some system denied access policies, therefore we don't have access to see the data inside. I'm w...
- 1694 Views
- 6 replies
- 0 kudos
- 0 kudos
You will need to backup the current metastore including the metadata and then start recreating the catalogs, schemas and tables on the new metastore.
- 0 kudos
- 1672 Views
- 10 replies
- 0 kudos
Update existing Metastores in AWS databricks
Hello Team,I am unable to update the Existing Metastore in my AWS databricks. I have new aws account and I am trying to update my existing workspace, however I am unable to update the s3 bucket details and Network configuration ( greyed out ) in the...
- 1672 Views
- 10 replies
- 0 kudos
- 0 kudos
Unfortunately there is no way to move the state of the workspace manually so based on this the solution will be to recreate the workspace and migrate the data
- 0 kudos
- 2012 Views
- 3 replies
- 1 kudos
How Can a Workspace Admin Grant Workspace Admin Permissions to a Group?
I want to grant Workspace Admin permissions to a group instead of individual users, but I haven’t found a way to do this. I considered assigning permissions by adding the group to the Databricks-managed 'admins' group (establishing a parent-child rel...
- 2012 Views
- 3 replies
- 1 kudos
- 1 kudos
No problem! I will check internally if there is any feature request of this nature. You can use the "admins" group for adding admin users or SPs.
- 1 kudos
- 1239 Views
- 2 replies
- 1 kudos
Resolved! DLT TABLES schema mapping
how we map table that in delta live table to a bronze , sliver , gold schema ? is that possible to store in different schema the dlt tables??
- 1239 Views
- 2 replies
- 1 kudos
- 965 Views
- 2 replies
- 0 kudos
AzureDevOps Repos Databricks update via pipeline not working
Hi all, im working with Azure DevOps and Databricks, using an app registration which it has permission on AzureDevOps and inside databricks as manager,user and in the group admins so it has permission over the repos.Im doing a pipeline to update or c...
- 965 Views
- 2 replies
- 0 kudos
- 0 kudos
Hello @TatiMun thanks for your question, can we review the following: Verify Remote URL: Double-check that the remote Git repo URL associated with the REPO_ID in Databricks is correct and accessible.Check PAT Permissions: Ensure that the Personal Acc...
- 0 kudos
- 2333 Views
- 2 replies
- 1 kudos
Security Consideration for OAUTH Secrets to use Service Principal to authenticate with Databricks
What are the security consideration we need to keep in mind when we want to us OAUTH Secrets to use a Service Principal to access Azure Databricks when Identity federation is disabled and workspace is not yet on boarded on to Unity Catalog? Can we co...
- 2333 Views
- 2 replies
- 1 kudos
- 1 kudos
Any updates on this?Also struggling with the OAuth security considerations. Specifically with updating the OAuth Secrets.Currently using a SP to access Databricks workspace for DevOps purposes through the Databricks CLI.I have the SP set up to renew ...
- 1 kudos
- 1596 Views
- 4 replies
- 0 kudos
Database Error in model Couldn't initialize file system for path abfss://
Recently the following error ocurs when running DBT:Database Error in model un_unternehmen_sat (models/2_un/partner/sats/un_unternehmen_sat.sql)Couldn't initialize file system for path abfss://dp-ext-fab@stcssdpextfabprd.dfs.core.windows.net/__unitys...
- 1596 Views
- 4 replies
- 0 kudos
- 0 kudos
Hi @Th0r ,Here is the explanation:Shallow clones in Databricks rely on references to data files of the original table. If the original table is dropped, recreated, or altered in a way that changes its underlying files, the shallow clone’s references ...
- 0 kudos
- 710 Views
- 0 replies
- 1 kudos
Databricks Asset Bundle a new way for amazing ETL
They say having the right tools at your disposal can make all the difference when navigating complex terrains. For organizations leveraging Databricks, simplifying deployment and scaling operations is often a key challenge.Over the years, I’ve explor...
- 710 Views
- 0 replies
- 1 kudos
- 3097 Views
- 2 replies
- 0 kudos
UserAgentEntry added to JDBC URL but not visible in Audit logs
Hi,As part of Databricks Best Practices, I have added 'UserAgentEntry' to JDBC URL that is being created when we are executing SQL statements through the JDBC driver.Sample url - jdbc:databricks://<host>:443;httpPath=<httpPath>; AuthMech=3;UID=token;...
- 3097 Views
- 2 replies
- 0 kudos
- 0 kudos
Sorry, I was mistaken. please ignore the previous response. The correct one isjdbc:databricks://<host>:443;httpPath=<httpPath>; AuthMech=3;UID=token;PWD=<token>;UserAgentEntry=<ApplicationName/Year>;
- 0 kudos
- 1833 Views
- 2 replies
- 1 kudos
Resolved! Exhausted Server when deploying a Databricks Assets Bundle (DAB)
Hello, I'm currently with a colleague inspecting the code and when trying to deploy the DAB it gets stuck: (.venv) my_user@my_pc my-dab-project % databricks bundle deploy -t=dev -p=my-dab-project-prod Building wheel... Uploading my-dab-project-...
- 1833 Views
- 2 replies
- 1 kudos
- 1 kudos
You are using a venv, the venv has too many files and is not needed to be included, try adding this on your databricks.ymlsync: exclude: - "venv" Hope it helps
- 1 kudos
- 1739 Views
- 3 replies
- 0 kudos
Delete the AWS Databricks account
I have created the aws databricks account from aws market place , but and I have cancelled the subscription after 14 days free trail from the market place. But still i see the account. How will i delete these databricks account associated with my ema...
- 1739 Views
- 3 replies
- 0 kudos
- 0 kudos
@dhruv1 As mentioned, it would be best to reach out to support for assistance.https://help.databricks.com/s/signuprequest
- 0 kudos
- 5805 Views
- 3 replies
- 1 kudos
Delta Live Table pipeline steps explanation
Does anyone have documentation on what is actually occurring in each of these steps?Creating update Waiting for resourcesInitializingSetting up tablesRendering graphFor example, what is the difference between initializing and setting up tables? I am ...
- 5805 Views
- 3 replies
- 1 kudos
- 1 kudos
Yes, loading data (full refresh/refresh) into all streaming tables and refreshing materialized views are part of the "Setting up table" step in a Delta Live Tables (DLT) pipeline when running in trigger mode.In triggered mode, materialized views are ...
- 1 kudos
- 1737 Views
- 1 replies
- 0 kudos
How to use Manged Identitify within Databricks Azure to access Blob Container?
Hi,My organization has asked that all blob storage accounts be accessed via managed identity. Several data brick notebooks are affected, so I'm currently trying to see how to set up a managed identity.We've added the Databricks resource provider to t...
- 1737 Views
- 1 replies
- 0 kudos
- 0 kudos
Have you followed the instructions available in docs https://learn.microsoft.com/en-us/azure/databricks/data-governance/unity-catalog/azure-managed-identities
- 0 kudos
- 4669 Views
- 1 replies
- 0 kudos
Resolved! Azure SCIM Provisioning Failures due to Resource Exhaustion
I had to make a significant change to the group membership in an Entra SCIM Provisioned group to Databricks, and the connector removed ALL users from the group (and sync'd) and then when I fixed, by adding a replacement dynamic group (with about 30% ...
- 4669 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @JSilverberg , This occurs when the number of requests goes beyond the rate limits mentioned here. https://learn.microsoft.com/en-us/azure/databricks/resources/limits#:~:text=Identity,No To prevent these issues in the future, it is best to do thes...
- 0 kudos
