I am ingesting job-cluster failure notifications that we send to OpsGenie into a delta table to automate the creation and tracking of Jira tickets. The alert notification includes the job run url, which we use to quickly respond to job failures. Ho...
I tracked this issue down to running the notebook on a shared Unity-enabled cluster. When scheduling the notebook to run on a job cluster with single user mode, this issue does not occur.
I have encountered with the Databricks CLI command databricks metastores list. As per our account setup, we have three metastores configured. However, when I run the command, it only returns information for one metastore instead of listing all three....
Hi @jv_v , Good Day!
The command you are running gives you Metastore details attached to your workspace. If you are looking to list out all the metastore present at the Account console, you can try running this below CLI command:
------------------...
Hello,I would like to know if there is direct access to the Databricks query history tables. For compliance issues, I would like to be able to create reports for something like: who has accessed a particular column in a table in the past 6 months. T...
Thanks @josh_melton ! I was wondering right now about this (one day after your post!) since I only found the UI and API in the documentation and was really puzzled that there is no equivalent in unity to the Snowflake query_history table.
SpoilerWe have a UC enabled on Databricks where we have two Metastores,Metastore in West Europe regionMetastore in North America regionAnd Databricks workspace.Workspace in West Europe regionAnd we want to have a single Databricks workspace for both ...
Hi,
Cross region metastore and workspace (WS) is not possible. It is important to understand the use case and why you have certain data in a region where you do not have a compute (WS) or do not need a compute. Ideally have each region has their own ...
Hello community !Here's my problem : I had a workspace with Unity Catalog enabled, and a catalog stored on an external location (Azure storage account). All my schemas and tables were storage on this storage. However we recently deleted the metastore...
Hi,
Looks like you are creating the catalog at the container level. Are there any other UC objects created pointing to that? Is the external location created at the container level? Can you please have a look at this and see if there is overlapping? ...
I have been using "rdd.flatMap(lambda x:x)" for a while to create lists from columns however after I have changed the cluster to a Shared acess mode (to use unity catalog) I get the following error: py4j.security.Py4JSecurityException: Method public ...
Hi All,My client has a table with user names and the authorized territory name the user has access to e.g EMEA, APAC etc. The users can also move to a different territory over time and the same will be updated in the user table table. I have multiple...
Hi,
Can you please have a look at this example please? https://docs.databricks.com/en/data-governance/unity-catalog/row-and-column-filters.html#mapping-table-examples
If you create account groups for each region and then use IS_ACCOUNT_GROUP_MEMBER t...
Message: Hi Databricks Community,I encountered an error while attempting to create a Metastore in Databricks. The error message is as follows:Error: cannot create metastore: This account with id ******************** has reached the limit for Metastor...
Hi!Recently we encountered a problem with how Databricks handles secrets that does not meet our compliance.We need total isolation of users credentials but seems like the admin role in databricks totally breaks that since a person with that role can ...
Let me put an example to make it more clear.We have a user, let's say Bob Rando, and he wants to store user and password in databricks secrets in order to use them later on the notebooks.Using the CLI Bob does the following:databricks secrets create-...
Setup UCX from Databricks web terminal
For cases when your desktop or laptop will not support the databricks labs install ucx technical requirements such as Python 3.10, administrative access, python package access or network access but your Databric...
Hi @Carol2, The vision of Unified Catalog (UC) and Purview in the context of government can vary depending on the specific use case and implementation. However, here are some insights:
Unified Catalog (UC):
UC is a data governance solution that h...
Currently, we must re-run our DLT pipelines anytime we want to add/modify a table or column description to unity catalog.Is this the only way to add/modify unity catalog descriptions?Is there functionality within databricks cli / sdk that we can leve...
Hi @wallfacer, Let’s explore some options for programmatically updating table or column descriptions in Unity Catalog within Databricks:
Databricks CLI and SDK:
You can leverage the Databricks Command-Line Interface (CLI) and the Databricks SDK t...
In my organization, we are using Databricks unity catalog and we have a metastore created for our region which holds all of our workspaces. When we created the metastore last year, we set a metastore root location for it (If I remember correctly, met...
Hello,What I did was that I isolated Root Storage from Workspaces (you can't reach storage from Network Perspective and Authentication Perspective) - it is ugly but it works.What you could try is to do that via API (like update metastore to NULL stor...
Hi all,I am getting an error when I try to 'Browse' an external location in catalog explorer through Databricks UI.This is the error I get: Error loading files.Input path url 'abfss://dbx-ccc-hive-metastore-unity-catalog@xxxxxxxxxx.dfs.core.windows.n...
Adding some tips...It seems that you have created a metastore with root path definition (metastore-level managed storage). If you're planning to deploy a multi-workspace environment (Development, QA and Production), I strongly recommend you to change...
Optimizing Your Site’s Branded Search - 5 Simple Ways Trusted by Every SEO Agency SydneyWhat is Branded searching Simply told, brand searches occur when individuals type your WP Creative in search engines like Google, Yahoo etc. Since customers are d...
Optimizing your site's branded search is crucial for enhancing visibility and driving organic traffic. Every SEO agency relies on several key strategies to achieve this goal effectively. Firstly, ensure your website's metadata, including title tags a...
