Here's your Data + AI Summit 2024 - Data Governance recap as you navigate the explosion of AI, data and tools in efforts to build a flexible and scalable governance framework that spans your entire data and AI estate. Keynote: Evolving Data Governan...
I need to know how much money each user spend in my workspace, i already have the system billing table, but there is not much information in it. How can i achive that?
You can tag the cluster and use this dashboard https://learn.microsoft.com/en-us/azure/databricks/admin/account-settings/usage
Hi @dataguy555 ,You will still need to have your external table in delta format. JSON or parquet are not supported:What is Delta Sharing? - Azure Databricks | Microsoft Learn
When attempting to write a dataframe to SQL Server using the provided "sqlserver" connector or the old "jdbc" one, it seems to be breaking due to old values used to build the dataframe. The code I'm using to write is:```python( df.write.format("sq...
Hi @MarthinusB, How are you doing today?To overcome this, Give a try explicitly defining the schema for your DataFrame to ensure no unwanted metadata is retained. You can also recreate the DataFrame using rdd.toDF() or checkpoint it to drop unnecessa...
I have a delta table with a column mask (azure databricks in eastus2 metastore) and another with no mask. I shared both tables with westeurope unity catalog metastore azure region via delta sharing (created share object, added the table as asset, ...
@prasadvaze That's correct, Delta Sharing does not work with RLS or column mask. It's well described in the documentation: https://docs.databricks.com/en/tables/row-and-column-filters.html#limitations
Currently, Databricks allows us to grant access to catalogs, schemas, and objects individually, but this process can be repetitive and time-consuming when managing access for multiple applications across different schemas. A feature that enables the...
Hi @sdheepak, Thanks for reaching out! Please review the response and let us know which best addresses your question. Your feedback is valuable to us and the community. If the response resolves your issue, kindly mark it as the accepted solution. Thi...
Hello,Is there any plan to support the equivalent of fabric semantic-link on Databricks? Essentially, having the ability to query PowerBi Dataset using SQL on interactive/jobs cluster and SQL warehouse (i.e a Unity Catalog federated source being Powe...
I think partners connect give option to connect BI tools and their sementic models to DBX not DBX to SM,in case you want to use models used for reports for AI/ML processing in dbx, you need to use something like semantic link to get data from SM dire...
We have an existing metastore in a region. We would like to delete it (Create a Unity Catalog metastore - Azure Databricks | Microsoft Learn) and create a new metastore in that same region. Are there any issues that we should be aware of with creatin...
I have exactly the same query.But before that, is it at all a good practice to delete the default metastore ? I am asking this because Databricks creates a lot of things on the Azure side like a storage account, access connector, managed identity, li...
Hi all,There is no official doc with a step-by-step process to enable Unity Catalog in azure databricks.If anyone has created doc or has the process, please share it here.Thanks, in advance.
Hi everyone!I have an Azure for Students subscription. Since I am unable to be an Azure Active Directory Global Administrator, I tried but it is restricted by the organization to add my user as a Global Administrator, I guess with an Azure for Studen...
Currently, we must re-run our DLT pipelines anytime we want to add/modify a table or column description to unity catalog.Is this the only way to add/modify unity catalog descriptions?Is there functionality within databricks cli / sdk that we can leve...
Comments for schemas and tables may be updated using SQL DDL statement COMMENT ON.Comments for columns are updated using: ALTER TABLE <table-name> ALTER COLUMN <column-name> COMMENT '<comment-text>'; The <column-name> for nested STRUCTS follows simp...
Hi!So I've been looking into trying Unity Catalog since it seems to add many great features.But one thing I cant get my head around is the fact that we cant (shouldn't?) use multiple metastores in the same region in UC.Let me explain my usecase:We ha...
I think the answer to this issue is have accounts by environment. Would be better if Databricks introduced an Organisations features as per AWS.
Message: Hi Databricks Community,I encountered an error while attempting to create a Metastore in Databricks. The error message is as follows:Error: cannot create metastore: This account with id ******************** has reached the limit for Metastor...
Even i am getting same error when i try to modify metastore with terraform script , i am using below script for same.( I have already the metastore in same region but i am not able to update the setting )resource "databricks_metastore" "example" { pr...
1: Has the RBAC Time based Grants been released?2: Can you provide an example of how to Setup a Time based Grant?3: Can Time based Grants be at a Time (HH:MM) and Date Level?
Hello everyone,I have just setup a Workspace and metastore in Azure Databricks, that by default enables Unity Catalog. I have defined several groups of users in our Account but when searching for that groups in the compute permissions dialog (no matt...
Hi, Sorry, I have already found the solution: from Databricks Account -> Workspaces -> open your specific workspace (Manage workspace) than add these already created groups with a role (User or Admin) in Permissions. Next, you'll be able to assign th...
Hi all,Is there any way to implement delta sharing in databricks without enabling the unity catalog?
Is it possible to receive data in databricks notebook without enabling the unity catalog?
Hi,The client secret recently expired and I updated the configuration with the new secret. From then on, I'm not able to access the data from ADF using the delta lake component, and here is the error message I keep getting:HTTP Error 401; url='https:...
Hi @saiV06, To resolve this, ensure you're using the correct client secret value, not the client secret ID. Verify the secret value, especially if it contains special characters, and ensure they are correctly encoded. Update the client secret in all ...
