Here's your Data + AI Summit 2024 - Data Governance recap as you navigate the explosion of AI, data and tools in efforts to build a flexible and scalable governance framework that spans your entire data and AI estate. Keynote: Evolving Data Governan...
Hi,In our Azure tenant We have deleted a Unity Catalog registered workspace and ticked the Delete default workspace storage permanentlyThere are 2 catalogs that were created in that workspace that are now greyed out in the catalog explorer when acces...
databricks catalogs delete --name my-catalog --force Thanks for the help ! Needed the force as the catalog was not empty. Also required a change of ownership of catalog to metastore owner.
we are using mount points via service principals approach to connect the storage account. and using the same mount points to create external tables in hive meta store. but now we are trying using only service principals setup ,so we need to change t...
Hi @mysura, Is your workspace Unity Catalog enabled? do you have a metastore assigned? I think legacy hive_metastore will not support service principal to do this change. I think the best approach is to create an external location based on the adfss ...
Is there a feature update request or resolution for adding service endpoints to the worker-vn that databricks created? We are experiencing 403 errors for IP authorizations from the worker-vn and do not have the permissions to update the service endpo...
In my workspace, I'm able to see the unity catalogs my team has created. I'm able to see the schemas and even query data in there and create views using the SQL Editor.When I go to write the same sql queries from a notebook using spark.sql or %sql I ...
i know this is an old topic, but i have the same issue with querying data using the sql editor and a sql warehouse. my sql warehouse does have unity catalog enabled. not sure why it wouldnt be working.
I've been followig this documentation to get Salesforce Data Cloud connection setup into Databrickshttps://learn.microsoft.com/en-us/azure/databricks/query-federation/salesforce-data-cloudI've added the client id and secret id and the scope, and it s...
Hi @KristiLogos , Good Day! We understand that you are facing the following error while you are trying to create a connection with Salesforce, but since it's been a long time, we wanted to check if you are still facing the issue or if it's resolved ...
Optimizing Your Site’s Branded Search - 5 Simple Ways Trusted by Every SEO Agency SydneyWhat is Branded searching Simply told, brand searches occur when individuals type your WP Creative in search engines like Google, Yahoo etc. Since customers are d...
HikeMyTraffic® provided excellent insights on Optimizing Your Site’s Branded Search. Their approach, along with strategies from SeoProfy and Livepage, made it easier to boost brand visibility. Highly recommend their expertise in enhancing branded sea...
I am trying to list OBO tokens that have been created for service principals. I tried using the Token Management API 2.0 (https://docs.databricks.com/dev-tools/api/latest/token-management.html#operation/get-tokens) to list workspace tokens, but it on...
Hi @Nick Tran​ Hope everything is going great.Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us so we ...
Watch the youtube video : https://www.youtube.com/watch?v=S9LLpMvAcT4
This is a great video! It's really helpful to see how to control access to catalogs in Unity Catalog. I especially like the part about granting read-only access to production data. This is a great way to allow data analysts to do their work without g...
Hi,As part of the DataGovernance or Authorization topic we are working on automation of the code for granting the access CATALOG LEVEL,SCHEMA LEVEL and TABLE LEVEL in Unity CatalogAs USE CATALOG Provides access at the Catalog level to user/group(whic...
USE is foundational. You need it at the catalog level and for any schema you want to access. So let's say I need SELECT only on 1 table T1 in Catalog:C1, Schema:S1 (Though there may be 1K tables in that schema) Then I would need USE CATALOG on C1;USE...
Managed tables are stored under the /user/hive/warehouse, which is also mentioned in the documentation. In our workflow, we use that path to read the parquet files from outside (through databricks connector). Can we assume this path is reliable, or i...
That path is reliable but we would recommend not using that path in general. That's your workspace root storage. Your data should be in a cloud path of your choosing (s3/adls/gcs) so that you can separate your data out by BU/Project/team etc based on...
I have a spreadheet containing table & column descriptions (comments) Is there a way to upload this against the schema in unity catalog? Basically instead of running 'Alter table <> alter column <> comment "description" ' command for every colum...
One of the ways to do it is to create a delta table using the Spreadsheet and loop through the delta table. Then, use the field value and create a dynamically alter query to update the description.
Hi,I have an external table which is created out of a S3 bucket. The first time I am creating the table I am using the following command : query = """CREATE TABLE IF NOT EXISTS catalog.schema.external_table_s3 USING PARQUET LOCAT...
Please try partition discovery for external tables. This feature should make it so that you can successfully run the MSCK REPAIR command, and more importantly, query external Parquet tables in a more performant way.
Hi,I tried testing by adding a Snowflake table through Databricks foreign catalog into a Data Clean Room. I was able to:Establish the connection.Query the data in Databricks successfully.However, I was unable to see the catalog when trying to add dat...
@RohithChippa Please check if you have the requirements to use clean room.To be eligible to use clean rooms, you must: Have an account that is enabled for serverless compute. See Enable serverless compute. Have a workspace that is enabled for Unity...
For some of my clusters i can able to find data in system.compute.node_timeline table to view utilization details even after 36 hrs since the run.
Yes seems that the runs are 1 minute of execution so this might be the reason why the metrics are not loaded
We try to migrate our old infra to Unity Catalog.We have some pipelines which write to BigQuery tables.To enable Unity Catalog to cluster level we have 2 options (Single user and Shared).Unfortunately we tried by using a Shared (Access mode) cluster ...
If you do a SHOW GRANTS on this cluster what does it shows? if you have SELECT on ANY File, then probably you just need to grant MODIFY on any file as described here https://docs.databricks.com/en/data-governance/table-acls/any-file.html
