Here's your Data + AI Summit 2024 - Data Governance recap as you navigate the explosion of AI, data and tools in efforts to build a flexible and scalable governance framework that spans your entire data and AI estate. Keynote: Evolving Data Governan...
HI Team,once after implementing unity catalog and start migrating external tables from legacy hive to unity catalog, i am seeing in articles that we need to make changes to our workloads to be in sync with 3 level name spaceeg: if i have 50 notebooks...
@Aviral Bhardwaj​ this will be normal priority and it takes lot of time to get response.team, we are looking for downtime possibilities and how to handle or change workflows or notebooks to use unity catalog 3 level namespace. do we have any utility ...
HiWe have faced an issue recently (everything worked well 1 week before)We are using exnernal hive metastore:"spark.hadoop.datanucleus.connectionPool.maxPoolSize" : "10" "spark.sql.hive.metastore.jars" : "/dbfs/ivyjars/*" "spark.sql.hive.metastor...
Generally, this should not be a case. Please raise a support request to databricks
I dont know what is TAClbut in the unity catalog you can have governance like catalog.database.table featurewhere you have to explicitly give access to the databricks data user so you have to give him access separately for catalog and database and ta...
We're trying to use Notebook widgets in DBR LTS 10.4 (Shared Mode) with Unity Catalogbut we're running into following error.Because of this our notebook dashboard are not working after migrating our data to Unity catalog.Following is the queryCREATE...
unity catalog is different and Data science panel is different , in the above notebook you have to use formatted or this widget will work there and in the unity catalog you have to use get argument formatted in the query
Zipped csv files are receiving to s3 raw layer.
@Jog Giri​ I also recently encountered a similar scenario, the below code solved my purpose without any issues.import zipfile for i in dbutils.fs.ls('/mnt/zipfilespath/'): with zipfile.ZipFile(i.path.replace('dbfs:','/dbfs'), mode="r") as zip_ref:...
I was trying to understand Unity Catalog, if metastore is at the account level or the each workspace in the account will have there own metastore? My understanding is the metastore cant be shared between two workspace.
@Sanjay Janardhan​ , no i have done this, you can share it in multiple workspaces
How prevent users from being able to create NEW databases? Possible without Unity Catalog?I'm working on access control and I know how to remove select and modify privileges on schemas and tables but that's it.
Hi @Christian Seberino​,Hope everything is going great.Thank you for reaching out to the community with your question. Would you be happy to mark an answer as best so that other members can find the solution more quickly?Cheers!
Hey guys,I am trying to create a delta live table in Unity Catalog as follows:CREATE OR REFRESH STREAMING LIVE TABLE <catalog>.<db>.<table_name> AS SELECT ... However, I get the error: org.apache.spark.sql.AnalysisException: Unsupported SQL statemen...
@Anant Pingle​ As of right now, DLT doesn't support Unity Catalog yet. We can expect this feature in upcomig days.
HiI have just created a metastore to allow unity catalog to be enabled for my workspace. However I am getting the following error when trying to create a table as per the unity catalog demo notbook:Error in SQL statement: UnauthorizedAccessException:...
Thanks for your answer to the second part of my question, super helpful! Can you confirm the first part of my question as well please?
Hello, I am trying to test the new federation for Snowflake feature in Databricks SQL, following the docs (https://docs.databricks.com/query-federation/snowflake.html).Environment:Subscription: PremiumCompute: DB SQL (serverless)Metastore: Unity Cata...
Hi @Landan George​, thanks for the quick response. I get the following error if I try to run the SQL code from a Notebook:SQLException: No suitable driver found for jdbc:snowflake://app.snowflake.com/region/foo/Any suggestions?
I could able to save data using pyspark into S3 but not sure on how to save a file stream object into S3 bucket using pyspark. I could achieve this with help of python but when Unity catalog was enabled on Databrciks it always ends up with an access ...
I got to know that there is a change required at Unity-catalog to make it work with Python and got a recommendation to use pyspark to store file into S3.I do not see much information about storing a file stream object in an S3 bucket anywhere. Can an...
How can we connect to unity catalog tables to create REST API's using Python/.NET?
I tried to create a view in hive_metastore.default which would access a table from a different catalog. Is there any chance to do so?eg.create view myTest as select * from someCatalog.someSchema.someTable
I will suggest the below links will help you, For exposing your Hive_metastore multiple catalogs, 1) Create 3 catalogs for each environment under single metastore by using Unity Catalog2) Now expose the Unity Catalog using DELTA SHARING to your BI ap...
We are setting up Databricks on our AWS account using customer managed VPC and enabling private link. I could see the private endpoints for frontend and backend with which we could spin up our EC2 instances. But the problem we are facing is to connec...
Este triángulo amoroso se puso complicado... ¡Faltan solo dÃas para el estreno de la segunda temporada de "Wildrose" en flixlatino!
