Here's your Data + AI Summit 2024 - Data Governance recap as you navigate the explosion of AI, data and tools in efforts to build a flexible and scalable governance framework that spans your entire data and AI estate. Keynote: Evolving Data Governan...
Hi,I want to access the Databricks Audit Logs to check the table usage information.I created a Databricks workspace on the premium pricing tier and enabled it for the Unity Catalogue.I configured Audit logs to be sent to Azure Diagnostic log delivery...
Thanks @Suteja Kanuri​ Could you guide me on how to setup and configure Table Access Control (TAC)?
We have a requirement to configure storage credentials in UC for s3 buckets which have been previously mounted by using access keys mounts has been depreciated and we need to configure storage credentials when we use UC. But while configuring storag...
Hello @karthik_p , The UC setup provides the feasibility to add the S3 bucket as a volume when compared to the non-UC concept Mounts.When added as Volume, the access to the Volume in UC is Governed via the Storage Credentials.Regarding Storage crede...
Execute the below Python code in your databricks workspace to enable lineage system tables import requests ctx = dbutils.notebook.entry_point.getDbutils().notebook().getContext() api_url = ctx.tags().get("browserHostName").get() api_token = ctx.apiTo...
Great, I wondered why I didn't see them
I’m currently trying to create a Foreign Catalog based on a Connection object of type SQLSERVER. This would allow me to directly access our on-premises MS SQL database from within Azure Databricks using Unity Catalog.As I’m part of a large organizati...
Good afternoon,We are in the process of setting up Unity Cat, we started in our existing workspace and set it up fine. We are particularly interested in federating existing warehouses at this point. We were able to federate one datawarehouse (dwh1) b...
Hi all,we are in the process of rolling out a new unity-enabled databricks env with 2 tiers: dev and prod.Initially we had the plan to completely decouple dev and prod, each with their own data lake as storage.While this is the safest option, it does...
That is the way I am working at right now. Assign workspace to the catalog and set to read-only if necessary.It would be easier though if it was possible to define a 2nd external location in read-only, as this cannot break anything (of course in rea...
Did any one tried UC DLT, we are trying to create multiple catalogs for UC DLTnote: as per limitations it won't support Locations, supports managed locationsOur use case: we want to create 2 catalogs one for dev and other for production. Dev catalog ...
We're setting up from scratch the Unity Catalog in our infrastructure in Azure, that is both- multi region ( europe, us)- multi env (dev, qa, prod)So, we did setup 2 metastore, one for each region, one in west europe and one for south central us.So f...
I'm the account admin for the subscription where the databricks workspace is created. However If I open the account console, its prompting me to select one of the databricks workspaces. Please note that its my own subscription via visual studio benef...
If there is an existing metastore for the region, doesn't mean my organization store all the meta data across the workspaces in a specific metastore or data lake ? Databricks suggests to have one metastore per region. How the project specific metadat...
When managing and safeguarding your internal data, data governance is crucial. It works like insurance, ensuring that all the data you gather is appropriately disseminated and kept safe within your company.Let's discuss data governance and how to imp...
We have setup the metastore with Manged Identity and when trying to create a managed table in the default location I am hitting below error. The storage is ADLS Gen2. AbfsRestOperationException: Operation failed: "This request is not authorized to pe...
Have you followed all the steps in here to create the metastore? https://app.getreprise.com/launch/wy1Y2ly/Do you have all the necessary permissions granted to create a Managed Table? https://docs.databricks.com/en/data-governance/unity-catalog/manag...
Hello,I am currently working with System Tables on Unity Catalogue part. I have loaded all the schemas in the catalogue and I am using PowerBI to directly access these tables. But while connecting PowerBI to Databricks, I am not able to see System Ta...
I had/have the same issue with the Tableau desktop. I'm not able to select the "Billing" schema because I don't see the "System" catalog. However, I found a workaround to resolve this issue. In databricks console, go to the "Catalog Explore" and sele...
Hi,Is it possible to make delta shares work for a Matlab client (Just the way we share delta tables to a PowerBI workbench)?We have Unity Catalog enabled and I wanted to explore more about data governance with aforesaid integration.Wanted to discuss ...
I'm trying to store MLlib instances in Unity Catalog Volumes. I think volumes are a great way to keep things organized.I can save to a volume without any issues and I can access the data using spark.read and with plain python open(). However, when I ...
Just to supplement that if the ML model is saved and then loaded within the same execution, calling load() will not cause the mentioned exception. Copying the model directory from UC volume to ephemeral storage attached to the driver node is also a w...
Hi All,I have created a new Unity Catalog (UC) metastore (new metastore ID) in west europe and now i want to import alle catalog/schema/tables form the old metastore (west europe) container to the new UC metastore.Note: coping all tables form the old...
Hi Barney,Just wondering if you got this issue solved.Thank you so much,Adriana Cavalcanti
