topic Re: Lakehouse Federation in Data Engineering

Lakehouse Federation

pwtnew32 — Tue, 21 Jan 2025 07:24:06 GMT

I use Lakehouse Federation to connect hive metastore (local VM) with metastore database type mysql

It's can see database and table in hive but when I query data session continue running without failed or success.

Do I have to migrate data to ADLs which mount with external location for catalog ?

Re: Lakehouse Federation

Avinash_Narala — Tue, 21 Jan 2025 09:32:19 GMT

As for Lakehouse Federation, only some sources are supported as of now. In that, connecting to HiveMetastore data which is in local VM is not supported, so you can migrate that data to ADLS and mount that as external location in unity catalog and query data from there.

Re: Lakehouse Federation

pwtnew32 — Thu, 23 Jan 2025 02:21:49 GMT

Thank you Avinash.
So in future, Will Lakehouse Federation support connection to hive metastore to query data on hive table in local ?

Re: Lakehouse Federation

Avinash_Narala — Thu, 23 Jan 2025 08:28:46 GMT

Maybe, but instead of focusing on connecting to local VM's I think Databricks will focus on broader audience, but will get to know once they announces more supported sources.

If my answer helps you, please accept it as solution.

Regards,

Avinash N