I have created an Azure data factory pipeline with a copy data function to copy data from adls path to a delta table .In the delta table drop downs i am able to see only the hive metastore database and tables only but the unity catalog tables are not...
Hi, I have the same issue.Additional information: the linked service created in azure data factory using azure databricks deltalake connector is using system-managed-identity rather than Token. Could we have an update? Thank you in advance.
Hi guys, how are you ?How can I access tables outside the Databricks ? (hive metastore)I have a python script in my local structure but I need to access are tables stored in Databricks (hive metastore) , how ???? Have any idea ?Thank you guys
Hi,I would like to clone the structure (schema only) of delta table without the content using SHALLOW CLONE command, as recommended in databricks documentation, but it doesn't work. I get an error message that this command is not supported in Unity C...
You can shallow clone the table to an external location after sync the result of the shallow clone to unity catalog. Here is an example https://medium.com/@wahidatoui/sync-delta-tables-stored-in-dbfs-managed-or-external-to-unity-catalog-1fe29a3ae61b
If I will migrate the Hive_metastore table into Unity Catalog table and subsequently hive metastore table is target of curation process. how I will migrate the delta record from Hive_metastore table to Unity Catalog table
Hi @Vivek Singh Banafar Hope everything is going great.Just checking in. Does @Debayan Mukherjee's response answer your question? If yes, would you be happy to mark it as best so that other members can find the solution more quickly?We'd love to he...
i have unit setup, and am trying to create a delta schema and table, i am getting an error on the schema creation, i am able to list the files and folders in the ADLS Gen2 storage account, i am able to write a parquet file to ADLS, but i cannot crea...
Hi @Shep Sheppard Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Than...
What is the most cost-effective way to sync delta tables across workspaces? I'd like to sync a production table from the production workspace to a staging workspace for testing. It doesn't have to be in real-time, on a daily basis is acceptable. We a...
Hi @Liz Zhang Hope everything is going great.Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us so we ...
Hi,we are creating an external table in the Azure databricks but facing this error:org.apache.spark.sql.AnalysisException: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.meta...
@Priyanka Biswas is there a place where we can have updates on metastore upgrades or control plane individual components. usually we can only see control plane region wide status
Hi @Sandhya SR, We haven’t heard from you since the last response from @Jensen Ackles and I was checking to see if their suggestions helped you.Or else, If you have any solution, please share it with the community, as it can be helpful to others.Al...
Hi Tried to create a delta table from spark data frame using below command:destination_path = "/dbfs/mnt/kidneycaredevstore/delta/df_corr_feats_spark_4"df_corr_feats_spark.write.format("delta").option("delta.columnMapping.mode", "name").option("path"...
Hi @Shafiul Alam ,who gave those names to columns? you can rename you columns, replace spaces / special characters, for example:%python
import re
list_of_columns = df_corr_feats_spark.colums
renamed_list_of_columns = [ re.sub(r'[^0-9a-zA-Z]+', "_", ...
