HiWe have faced an issue recently (everything worked well 1 week before)We are using exnernal hive metastore:"spark.hadoop.datanucleus.connectionPool.maxPoolSize" : "10"
"spark.sql.hive.metastore.jars" : "/dbfs/ivyjars/*"
"spark.sql.hive.metastor...
I dont know what is TAClbut in the unity catalog you can have governance like catalog.database.table featurewhere you have to explicitly give access to the databricks data user so you have to give him access separately for catalog and database and ta...
We're trying to use Notebook widgets in DBR LTS 10.4 (Shared Mode) with Unity Catalogbut we're running into following error.Because of this our notebook dashboard are not working after migrating our data to Unity catalog.Following is the queryCREATE...
unity catalog is different and Data science panel is different , in the above notebook you have to use formatted or this widget will work there and in the unity catalog you have to use get argument formatted in the query
@Jog Giri​ I also recently encountered a similar scenario, the below code solved my purpose without any issues.import zipfile
for i in dbutils.fs.ls('/mnt/zipfilespath/'):
with zipfile.ZipFile(i.path.replace('dbfs:','/dbfs'), mode="r") as zip_ref:...
I was trying to understand Unity Catalog, if metastore is at the account level or the each workspace in the account will have there own metastore? My understanding is the metastore cant be shared between two workspace.
How prevent users from being able to create NEW databases? Possible without Unity Catalog?I'm working on access control and I know how to remove select and modify privileges on schemas and tables but that's it.
Hi @Christian Seberino​,Hope everything is going great.Thank you for reaching out to the community with your question. Would you be happy to mark an answer as best so that other members can find the solution more quickly?Cheers!
Hey guys,I am trying to create a delta live table in Unity Catalog as follows:CREATE OR REFRESH STREAMING LIVE TABLE <catalog>.<db>.<table_name> AS SELECT ... However, I get the error: org.apache.spark.sql.AnalysisException: Unsupported SQL statemen...
HiI have just created a metastore to allow unity catalog to be enabled for my workspace. However I am getting the following error when trying to create a table as per the unity catalog demo notbook:Error in SQL statement: UnauthorizedAccessException:...
Hello, I am trying to test the new federation for Snowflake feature in Databricks SQL, following the docs (https://docs.databricks.com/query-federation/snowflake.html).Environment:Subscription: PremiumCompute: DB SQL (serverless)Metastore: Unity Cata...
Hi @Landan George​, thanks for the quick response. I get the following error if I try to run the SQL code from a Notebook:SQLException: No suitable driver found for jdbc:snowflake://app.snowflake.com/region/foo/Any suggestions?
I could able to save data using pyspark into S3 but not sure on how to save a file stream object into S3 bucket using pyspark. I could achieve this with help of python but when Unity catalog was enabled on Databrciks it always ends up with an access ...
I got to know that there is a change required at Unity-catalog to make it work with Python and got a recommendation to use pyspark to store file into S3.I do not see much information about storing a file stream object in an S3 bucket anywhere. Can an...
Hi @Sandhya SR​, We haven’t heard from you since the last response from @Jensen Ackles​ and I was checking to see if their suggestions helped you.Or else, If you have any solution, please share it with the community, as it can be helpful to others.Al...
I tried to create a view in hive_metastore.default which would access a table from a different catalog. Is there any chance to do so?eg.create view myTest as select * from someCatalog.someSchema.someTable
I will suggest the below links will help you, For exposing your Hive_metastore multiple catalogs, 1) Create 3 catalogs for each environment under single metastore by using Unity Catalog2) Now expose the Unity Catalog using DELTA SHARING to your BI ap...
We are setting up Databricks on our AWS account using customer managed VPC and enabling private link. I could see the private endpoints for frontend and backend with which we could spin up our EC2 instances. But the problem we are facing is to connec...
Hi @Rahul Mishra​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Thank...
