Here's your Data + AI Summit 2024 - Data Governance recap as you navigate the explosion of AI, data and tools in efforts to build a flexible and scalable governance framework that spans your entire data and AI estate. Keynote: Evolving Data Governan...
I am currently working on syncing workspaces in Databricks Accounts console, and I came across the command databricks-labs-ucx sync-workspace-info. I am wondering if this command is suitable for both UCX (Unity Catalog) assessments and other non-UCX ...
Hi there, you can use UCX sync to get an inventory of everything in your workspaces and track it as you upgrade to avoid cross workspace duplication where there's not a 1:1 mapping. It's designed for Unity Catalog upgrades, so whilst it's battle test...
Hi Databricks Community,I am currently working on integrating SCIM provisioning for user and group management in Databricks. I need to configure the SCIM endpoint and token for automated user provisioning.Could you please confirm if there is a specif...
Hi @Padmaja, Thanks for reaching out! Please review the response and let us know if it answers your question. Your feedback is valuable to us and the community. If the response resolves your issue, kindly mark it as the accepted solution. This will h...
Setup UCX from Databricks web terminal For cases when your desktop or laptop will not support the databricks labs install ucx technical requirements such as Python 3.10, administrative access, python package access or network access but your Databric...
Hi @DouglasMoore,I would like to provide some clarification regarding the commands mentioned in your post about DBR 15.1 or above. Contrary to what was stated, these commands are indeed necessary. Recently, when attempting to install UCX via the web ...
no information on character_maximum_length and is_identity in unity catalog table system.information_schema.columnstable definition exampleCREATE TABLE tyable_name ( UniqueID bigint not null GENERATED ALWAYS AS IDENTITY , ID varchar(256))looks like ...
Hello Team,Still i am facing same issue of value zero in character_maximum_length. Is there any other way we can get character_maximum_length of columns?When we can expect to resolve this bug?
Hello Databricks Community,I'm seeking advice and insights on best practices for managing data access and permissions in Databricks. Our company currently uses a view-based approach for access control, but I'm wondering if we should transition to Dat...
Hi @jackgurae, Managing data access and permissions in Databricks involves evaluating two main approaches: a view-based method versus leveraging Databricks' native security features. The view-based approach offers simplicity by abstracting complex lo...
Hi everyone,I need to transfer tables present in the Hive metastore from one workspace to another. My ultimate goal is to migrate these tables to Unity Catalog. I already have a code that will handle the migration to Unity Catalog, but I need help wi...
Are these tables managed tables or external tables? This is very important step for your migration.
Hello Databricks Community,I am currently facing an issue while trying to export and import all catalogs from an old workspace to a newly created workspace using Databricks' REST API and Python. Here's a summary of my situation:Objective: I am attemp...
Hi everyone,I wanted to follow up on my previous question about exporting hive_merastore catalog from existing workspace and import it to a new workspace along with all its dependencies using rest api ,if not other alternative solution how to import ...
Hello friends! Do we have any certifications for Unity Catalog in Databricks?
@Techelligence You can find the Platform Administrator path from here: https://customer-academy.databricks.com//lms/index.php?r=coursepath/deeplink&id_path=207&hash=1003b93351e6a5abe05aa342f12b1458f6ac2799&generated_by=714319There are many other cert...
Is there any way of hide definition of view in unity catalog?
Hi @mahfooz_iiitian ,Currently there is no such an option in unity catalog.
I'm creating a unity catalog using terraform on Azure. The first couple of apply worked just fine, no error at all, but suddenly today when i wanted to incorporate another change to the code which was not related to databicks, terraform started givin...
Hi @JustLeo , You haven't provided how you defined resource, but maybe you're missing depends on clause?According to terraform documentation:In Terraform 0.13 and later, data resources have the same dependency resolution behavior as defined for m...
I have created an Azure data factory pipeline with a copy data function to copy data from adls path to a delta table .In the delta table drop downs i am able to see only the hive metastore database and tables only but the unity catalog tables are not...
Hi all, Please try following this syntax in the database field <catalog_name>`.`<database_name> It is important that you use ` not ' This should force the connect to use a Unity Catalog Database.
Here's your Data + AI Summit 2024 - Data Governance recap as you navigate the explosion of AI, data and tools in efforts to build a flexible and scalable governance framework that spans your entire data and AI estate. Keynote: Evolving Data Governan...
We've an on-premises DB2/LUW and Z/Os database and we need to show a lineage of them. Can Unity Catalog or Purview do this? If we've the best, what would it be? Or would working with both be a better option?
Hi @rjsilva1987, Both Unity Catalog and Purview can help you with data lineage, but let’s explore the specifics: Unity Catalog: Unity Catalog is part of Databricks’ data governance and metadata management solution.It provides data lineage capabil...
I'm facing an issue while trying to create an external location in Databricks using Terraform and a service principal. The specific error message I'm encountering is:Error:Here's some context:Using Azure CLI (Az login): The creation of the external l...
After creating the databricks_metastore resource, can you run databricks_grants? like this resource "databricks_grants" "foo" { depends_on = databricks_metastore.foo metastore = databricks_metastore.foo.id grant { principal = <your service ...
I'm encountering an issue with my Terraform code for creating a cluster. The terraform plan command runs successfully and shows the correct changes, but the after that fails with errors. Here are the details: Terraform Code:terraform {required_provid...
Are you getting two different errors?default auth error usually means you you need to explicitly set the providers in either the data or resource objects as well, or you're missing a depends_on attribute. I think for both cases it is the latter. i.e....
