Here's your Data + AI Summit 2024 - Data Governance recap as you navigate the explosion of AI, data and tools in efforts to build a flexible and scalable governance framework that spans your entire data and AI estate. Keynote: Evolving Data Governan...
In my workspace, I'm able to see the unity catalogs my team has created. I'm able to see the schemas and even query data in there and create views using the SQL Editor.When I go to write the same sql queries from a notebook using spark.sql or %sql I ...
i know this is an old topic, but i have the same issue with querying data using the sql editor and a sql warehouse. my sql warehouse does have unity catalog enabled. not sure why it wouldnt be working.
I've been followig this documentation to get Salesforce Data Cloud connection setup into Databrickshttps://learn.microsoft.com/en-us/azure/databricks/query-federation/salesforce-data-cloudI've added the client id and secret id and the scope, and it s...
Hi @KristiLogos , Good Day! We understand that you are facing the following error while you are trying to create a connection with Salesforce, but since it's been a long time, we wanted to check if you are still facing the issue or if it's resolved ...
Optimizing Your Site’s Branded Search - 5 Simple Ways Trusted by Every SEO Agency SydneyWhat is Branded searching Simply told, brand searches occur when individuals type your WP Creative in search engines like Google, Yahoo etc. Since customers are d...
HikeMyTraffic® provided excellent insights on Optimizing Your Site’s Branded Search. Their approach, along with strategies from SeoProfy and Livepage, made it easier to boost brand visibility. Highly recommend their expertise in enhancing branded sea...
I am trying to list OBO tokens that have been created for service principals. I tried using the Token Management API 2.0 (https://docs.databricks.com/dev-tools/api/latest/token-management.html#operation/get-tokens) to list workspace tokens, but it on...
Hi @Nick Tran​ Hope everything is going great.Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us so we ...
Watch the youtube video : https://www.youtube.com/watch?v=S9LLpMvAcT4
This is a great video! It's really helpful to see how to control access to catalogs in Unity Catalog. I especially like the part about granting read-only access to production data. This is a great way to allow data analysts to do their work without g...
Hi,As part of the DataGovernance or Authorization topic we are working on automation of the code for granting the access CATALOG LEVEL,SCHEMA LEVEL and TABLE LEVEL in Unity CatalogAs USE CATALOG Provides access at the Catalog level to user/group(whic...
USE is foundational. You need it at the catalog level and for any schema you want to access. So let's say I need SELECT only on 1 table T1 in Catalog:C1, Schema:S1 (Though there may be 1K tables in that schema) Then I would need USE CATALOG on C1;USE...
Managed tables are stored under the /user/hive/warehouse, which is also mentioned in the documentation. In our workflow, we use that path to read the parquet files from outside (through databricks connector). Can we assume this path is reliable, or i...
That path is reliable but we would recommend not using that path in general. That's your workspace root storage. Your data should be in a cloud path of your choosing (s3/adls/gcs) so that you can separate your data out by BU/Project/team etc based on...
I have a spreadheet containing table & column descriptions (comments) Is there a way to upload this against the schema in unity catalog? Basically instead of running 'Alter table <> alter column <> comment "description" ' command for every colum...
One of the ways to do it is to create a delta table using the Spreadsheet and loop through the delta table. Then, use the field value and create a dynamically alter query to update the description.
Hi,I have an external table which is created out of a S3 bucket. The first time I am creating the table I am using the following command : query = """CREATE TABLE IF NOT EXISTS catalog.schema.external_table_s3 USING PARQUET LOCAT...
Please try partition discovery for external tables. This feature should make it so that you can successfully run the MSCK REPAIR command, and more importantly, query external Parquet tables in a more performant way.
Hi,I tried testing by adding a Snowflake table through Databricks foreign catalog into a Data Clean Room. I was able to:Establish the connection.Query the data in Databricks successfully.However, I was unable to see the catalog when trying to add dat...
@RohithChippa Please check if you have the requirements to use clean room.To be eligible to use clean rooms, you must: Have an account that is enabled for serverless compute. See Enable serverless compute. Have a workspace that is enabled for Unity...
For some of my clusters i can able to find data in system.compute.node_timeline table to view utilization details even after 36 hrs since the run.
Yes seems that the runs are 1 minute of execution so this might be the reason why the metrics are not loaded
We try to migrate our old infra to Unity Catalog.We have some pipelines which write to BigQuery tables.To enable Unity Catalog to cluster level we have 2 options (Single user and Shared).Unfortunately we tried by using a Shared (Access mode) cluster ...
If you do a SHOW GRANTS on this cluster what does it shows? if you have SELECT on ANY File, then probably you just need to grant MODIFY on any file as described here https://docs.databricks.com/en/data-governance/table-acls/any-file.html
Hi All,A question for those using Delta Sharing, how are you defining naming conventions for Shares and Recipient names? What best practices or standards are you following? Thanks, Fatima
For platform-to-platform environments, I nest all shared tables under a specially named Schema with a prefix of DS-. For example, my schema name could be ds-mfgschema. This is a 'flag' that any tables nested under this schema are being delta shared ...
I am exploring the search capabilities available in Unity Catalog. Specifically, I want to know if it is possible for a user to discover the existence of a table and its owner via search, even if they don't have access to the table's data. This featu...
I just found out that with the BROWSE privilege on the catalog this can be done
Due to some problems, I need to set Py4JSecurity to false, however I get the following message: "spark.databricks.pyspark.enablePy4JSecurity is not allowed when choosing an access mode"My cluster is in Shared access mode and a runtime version 12.1 (i...
Hello, any update on this? Is this still something not allowed in Databricks? I need to run a function that is not whitelisted and I need that function to run on a shared cluster as I need to read from a table that has a masking in it. So I need eith...
