Urgent: Installing Lakebridge on Databricks

[aditi_mokashi]

Hi,

I want to install Databricks Lakebridge on my Databricks environment and use the analyze and transpile commands through a python script.

The usecase is that we need to create an automated pipeline that will migrate the existing scripts from snowflake to databricks at a scheduled interval. The pipeline should first - fetch all DDLs from snowflake, then it should migrate it using Lakebridge and then the migrated scripts should be executed to create tables and functions.

I can get the DDLs from snowflake and dump it in unity catalog as .sql files but I am not able to run the Lakebridge on Databricks environment, I tried using the Web Terminal for Serverless compute but lakebridge installation fails with error:
"Error: python: create venv: /usr/bin/python3.12 -m venv /home/spark-***-***-***-***-***/.databricks/labs/lakebridge/state/venv: exit status 1"

I can use "subprocess" module of Python to run the Lakebridge commands but I cannot figure out how to install Lakebridge.

Please tell me if this scenario is even possible, any help is appreciated!

Please note that I have tried using Lakebridge on my local machine but my requirement is to create a pipeline for it, please help me with this.

Thanks in advance!

[yogeshsingh]

Thanks for the detailed write-up. Short answer: your workflow is possible, but installing/running Lakebridge from a Databricks serverless web terminal is not the recommended path today.

The error you’re seeing happens because Lakebridge installation expects to create its own Python virtual environment, and serverless web terminal environments are too restricted for that in many cases.

What should work instead:

• export the Snowflake DDL/SQL files

• run Lakebridge from a machine you control (local machine, VM, jump box, or container runner)

• write the converted output back to Databricks

• use a Databricks job/workflow to execute the migrated scripts afterward

A few notes:

• Your Snowflake → Lakebridge → Databricks flow is a valid use case.

• Snowflake is supported as a transpile source.

• If you want to run Lakebridge inside Databricks anyway, the better workaround is a dedicated all-purpose / single-user cluster rather than serverless web terminal.

So my recommendation is: keep the Lakebridge runtime outside the restricted serverless environment, and use Databricks for orchestration and execution of the converted output.

[iyashk-DB]

Hi, the error you're seeing is consistent with the execution environment rather than with Lakebridge itself.

Lakebridge is a Databricks CLI Labs extension (databricks labs install lakebridge) designed to run wherever the Databricks CLI is installed—your local machine, a VM, a container, or a CI runner—not within workspace compute.

It requires Python 3.10–3.14, Java 21+, and outbound connectivity to GitHub, Maven Central, and PyPI to download its dependencies. The serverless web terminal is a managed, restricted environment that isn't intended for this workflow, so the virtual environment creation failure is expected and isn't something that can typically be resolved by changing the Python environment.

For the workflow you described, I'd recommend moving the Lakebridge conversion step outside Databricks entirely. Run Lakebridge on a CI runner, VM, or container with the required network access and Java installation, use it to retrieve and convert the Snowflake DDL, then upload the converted output into your Databricks workspace using the CLI or REST APIs. A Databricks Job can then execute the generated artifacts.

That separation aligns with how the tooling is intended to be used: Lakebridge performs the conversion, while Databricks executes the converted code.