problem with workspace after metastore deleted

sg-vtc — Wed, 18 Oct 2023 00:44:49 GMT

I am completely new to Databricks AWS and start working on it a week ago. Pls excuse me if I ask or did something silly.

I created a workspace and a single node cluster for testing.
A metastore was created from Databricks quickstart and it was automatically assigned to the workspace which I did not realize. I thought the workspace is using default metastore from dbfs.

I did some basic testing using this database and able to create external delta tables using cvs file from non-aws object storage. This is my main goal of test to ensure databricks can integrate with non-aws s3 compatible object storage.

However, from time to time I saw this error:

parent external location for path 's3://databricks-workspace-stack-xxxxxx-metastore-bucket/b11fb1a0-a462-4dfb-b91b-e0795fde10b0’ does not exist.
From my aws account, I can see the bucket databricks-workspace-stack-xxxxxx-metastore-bucket created by Databricks quickstart with appropriate IAM role and policy. However, I did not see any folder or object in this bucket.

I created this b11fb1a0-a462-4dfb-b91b-e0795fde10b0 folder in the S3 metastore bucket but still get the error re the parent external location does not exist.

Since I did not see object/folder in this bucket, I deleted the metastore from Databricks console. After that I start having problem with the workspace since it did not have any metastore assigned to it.

I created another s3 bucket and follows the naming convention and IAM role/policy that was created by Databricks before for databricks-workspace-stack-xxxxxx-metastore-bucket

I created the new Databricks metastore, when assigned it to the workspace that I am using. From the workspace details, I can see the new metastore path. However, when I run some python commands in this workspace, I always get this error, it is still expecting the previous metastore that was assigned but deleted by me.

ErrorClass=METASTORE_DOES_NOT_EXIST] Metastore 'b11fb1a0-a462-4dfb-b91b-e0795fde10b0' does not exist.

How to resolve the problem. I prefer not to delete my existing workspace.

Any help is greatly appreciated.

Re: problem with workspace after metastore deleted

sg-vtc — Wed, 18 Oct 2023 18:11:54 GMT

I restarted the compute node and this problem went away.
ErrorClass=METASTORE_DOES_NOT_EXIST] Metastore 'b11fb1a0-a462-4dfb-b91b-e0795fde10b0' does not exist.

New question: I am testing Databricks with non-AWS S3 object storage. I can access the non-AWS S3 bucket by setting these parameters:

sc._jsc.hadoopConfiguration().set("fs.s3a.access.key", "XXXXXXXXXXXXXXXXXXXX")
sc._jsc.hadoopConfiguration().set("fs.s3a.secret.key", "XXXXXXXXXXXXXXXXXXXXXXXXXXXX")
sc._jsc.hadoopConfiguration().set("fs.s3a.endpoint", "XXXXXXXXXXXX.com")

I can read the csv files in the bucket

spark.read.format("csv").option("inferschema","true").option("header","true").option("sep","|").load("s3://deltalake/10g_csv/reason.csv")

When trying to create external table from this csv, got AWS Security token service invalid error. Since I am not using AWS s3 bucket, is there a way to skip this checking.

I can see Databricks created parquet file and _delta_log folder in this external bucket location but it did not complete the delta table creation. It did not create 00000000000000000000.crc and 00000000000000000000.json in the _delta_log folder.

topic Re: problem with workspace after metastore deleted in Get Started Discussions

problem with workspace after metastore deleted

Re: problem with workspace after metastore deleted