I'm trying to use bamboolib on the 11.0 recommended DBR. I am using the option "Databricks: Load database table" The databases and tables that are on the unity catalog do not populate the dropdown boxes. How can I fix this?
Hello @k.b.​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Thanks!
We are looking to send messages using databricks to an azure service bus topic and would like to connect to the service bus using a managed identity vs a connection string. Is this possible in databricks. The only thing I could find regarding datab...
To be honest, the Databricks team sucks. They should add a simple Identity tab under the Databricks workspace resource as it is done for other Microsoft services. They don't think about making product easier to maintain and configure, they look from ...
I have a unity catalog which is binded to multiple workspaces (Attached to Metastore), In both workspaces I have created different tables under this catalog will the Metastore admin will have access to these tables and data in those tables in differe...
Unity Catalog acts as a single point of access for managing data across Databricks workspaces, which means with unity catalog a Metastore admin can manage the access permissions on the Data which comes under Unity catalog umbrella.Data inside differe...
Is it possible to provide fine grained control (folder level/file level) for a given volume?I have two SCIM integrated groups who have read volume access at the catalog level, but those two groups need different permissions on a lower level. Preferab...
Can you define the external location at the Landing level and create two Volumes one for PDF and other for CSV and provide access to the respective groups 1 and 2.
I have been using "rdd.flatMap(lambda x:x)" for a while to create lists from columns however after I have changed the cluster to a Shared acess mode (to use unity catalog) I get the following error: py4j.security.Py4JSecurityException: Method public ...
Faced this issue multiple times.Solution:1. Don't use Shared Cluster or cluster without Unity Catalog enabled for running 'rdd' queries on Databricks.2. Instead create a Personal Cluster (Single User) with basic configuration and with Unity Catalog e...
All data finish in Excel, but do it with some dignity. Official #databricks dbsqlcli allows it to be done in the old-style mysqldump way.https://docs.databricks.com/en/dev-tools/databricks-sql-cli.html
We're setting up from scratch the Unity Catalog in our infrastructure in Azure, that is both- multi region ( europe, us)- multi env (dev, qa, prod)So, we did setup 2 metastore, one for each region, one in west europe and one for south central us.So f...
Hi @databriccone, Setting up Unity Catalog in your Azure infrastructure is a crucial step for better data governance and operational efficiency. Let’s address your questions:
Naming Conventions:
For naming conventions, consider adding a suffix li...
I saw a really nice article (https://www.databricks.com/blog/announcing-public-preview-volumes-databricks-unity-catalog) on the incorporation of various elements of data lineage within Unity Catalog. In my own exploration, I've been able to replicate...
Hi @23940829381, Thank you for sharing your interest in data lineage within Unity Catalog! It’s a powerful feature that allows you to track the flow of data across various elements in your Databricks environment.
Let’s delve into this further:
Un...
I wanted to set description of a schema in Unity Catalog.Description was greyed out and on pencil icon I got: "Requires Ownership of SCHEMA 'xxxxx'."Owner is set to the Azure Active Directory group that I am member of.I was able to change the owner t...
AAD group membership cannot be used as the object owner in UC to edit object description directly. Object ownership typically refers to individual user accounts, not group memberships, in most systems.
In my organization, we are using Databricks unity catalog and we have a metastore created for our region which holds all of our workspaces. When we created the metastore last year, we set a metastore root location for it (If I remember correctly, met...
Hello,What I did was that I isolated Root Storage from Workspaces (you can't reach storage from Network Perspective and Authentication Perspective) - it is ugly but it works.What you could try is to do that via API (like update metastore to NULL stor...
We're thrilled to introduce UCX v0.15.0, packed with cutting-edge features, enhancements, and deprecations that'll take your upgrading experience to new heights!
Release Highlights:
AWS S3 support: We've added AWS S3 support to the migrate-location...
I noticed in the Catalog Explorer of an Unity Catalog integrated workspace that there is a default catalog named 'system' owned by 'System user'. Who is this system user? It is not listed in the admin dashboard of all workspace users.
Hi @BobBubble2000, The ‘System user’ in the context of Unity Catalog within an Azure Databricks workspace plays a crucial role in managing data governance and access control.
Unity Catalog:
Unity Catalog is a fine-grained governance solution for d...
We are attempting to create a comprehensive Data Governance solution using Unity Catalog & INFA CDGC tool. The objective is to onboard the business assets on the Informatica DG platform and use the unity catalog to trace technical assets and lineage ...
Yes, Informatica now has native integration to Unity Catalog with its cloud data catalogue and governance service. You can scan Unity catalog Metadata and propagate this in CDGC and apply policies, business terms, overlay data quality scores, display...
Hi all,I try to create an external volume in a catalog/schema on a subpath of the external location.This works fine, however for a certain path I cannot create an external volume. I always getInput path url overlaps with managed storage within 'Crea...
I have the same issue. I have created a Catalog using External Location, deleted it and now i can't create Volume for this Location even thought Catalog is deleted long ago
I am using Unity Catalog to store my models. My Unity Catalog meta-store is hosted in an Azure Storage Account, which has public network access disabled. Access is restricted to certain IP ranges and private endpoint connections only.Recently, my dev...
I have the same problem. It must be a permissions issue. In my case, it happens when the models are under a schema that was created without indicating a specific "MANAGED LOCATION". In those cases, I also have problems trying to load a model. When th...
