topic Re: Terraform: get metastore id without creating new metastore in Data Engineering

Terraform: get metastore id without creating new metastore

LukaszJ — Mon, 14 Mar 2022 11:15:00 GMT

Hello,

I want to create database (schema) and tables in my Databricks workspace using terraform.

I found this resources: databricks_schema

It requires databricks_catalog, which requires metastore_id.

However, I have databricks_workspace and I did not create any metastore and everything works fine. So how can I get metastore_id without creating a new mestastore, or how can I create databricks_schema (database) other way?

Best regards,

Łukasz

Re: Terraform: get metastore id without creating new metastore

Hubert-Dudek — Mon, 14 Mar 2022 12:45:44 GMT

@Łukasz Jaremek , Strange with that id. For example from databricks cluster log you can get some info about metastore but not id... (maybe url can be used instead?)

example from log:

22/03/14 05:40:38 INFO MetastoreMonitor$: Internal internal metastore configured (config=DbMetastoreConfig{host=consolidated-westeurope-prod-metastore-addl-3.mysql.database.azure.com, port=3306, dbName=organization34353553, user=LoBp1K7HeU3FvyzM@consolidated-westeurope-prod-metastore-addl-3})
22/03/14 05:40:38 INFO DatabricksILoop$: Creating throwaway interpreter
22/03/14 05:40:38 INFO DataSourceFactory$: DataSource Jdbc URL: jdbc:mariadb://consolidated-westeurope-prod-metastore-addl-3.mysql.database.azure.com:3306/organization34353553?useSSL=true&sslMode=VERIFY_CA&disableSslHostnameVerification=true&trustServerCertificate=false&serverSslCert=/databricks/common/mysql-ssl-ca-cert.crt

Re: Terraform: get metastore id without creating new metastore

Atanu — Sun, 17 Apr 2022 17:41:09 GMT

https://registry.terraform.io/providers/databrickslabs/databricks/latest/docs/resources/schema I think this is for UC. https://docs.databricks.com/data-governance/unity-catalog/index.html

Re: Terraform: get metastore id without creating new metastore

LukaszJ — Fri, 29 Apr 2022 06:21:18 GMT

Thank you for the answer.

Yes, that's probably it. I have to read about the UC.

Best regards,

Łukasz

Re: Terraform: get metastore id without creating new metastore

LukaszJ — Fri, 29 Apr 2022 06:24:26 GMT

Hello @Kaniz Fatma , thank you for your interest.

I think @Atanu Sarkar solved my problem.

I'm sorry for the lack of response.

Best reagards,

Łukasz