topic Re: Redshift Stored Procedure Migration to Databricks in Data Engineering

Redshift Stored Procedure Migration to Databricks

Avinash_Narala — Thu, 30 Jan 2025 07:59:10 GMT

Hi,

I want to migrate Redshift SQL Stored Procedures to databricks.

As databricks doesn't support the concept of SQL Stored Procedures.

How can I do so?

Re: Redshift Stored Procedure Migration to Databricks

MariuszK — Thu, 30 Jan 2025 08:59:54 GMT

You can create a Python function or move logic a notebook, it depend what is the purpose of the procedure.

Re: Redshift Stored Procedure Migration to Databricks

Avinash_Narala — Thu, 30 Jan 2025 12:21:20 GMT

i have a bunch of procedures to convert. so it is not possible to generalise the purpose of each procedure.

I need a wrapper program to convert redshift sql procedures into python functions in databricks notebooks

Re: Redshift Stored Procedure Migration to Databricks

MariuszK — Thu, 30 Jan 2025 12:33:31 GMT

You can try with ChatGPT, ask the chat to convert it, you will be really suppressed. There is more advance option for that, but it's more complex.

Re: Redshift Stored Procedure Migration to Databricks

Rjdudley — Thu, 30 Jan 2025 14:12:44 GMT

I am assuming you want a sproc so you have some code which is reusable across many notebooks. If so, Databricks supports several types of functions: https://learn.microsoft.com/en-us/azure/databricks/sql/language-manual/sql-ref-functions

However if your Redshift sprocs are all of the processing logic, you should convert those to notebooks instead.

I don't know of any program which will directly convert Redshift SQL to Python (or Databricks SQL). An LLM might help, perhaps even the Databricks Assistant in the cloud IDE.

Re: Redshift Stored Procedure Migration to Databricks

Rjdudley — Wed, 05 Feb 2025 17:02:50 GMT

Hey @Avinash_Narala in case you didn't see this announcement, looks like it might solver your problem: Welcoming BladeBridge to Databricks: Accelerating Data Warehouse Migrations to Lakehouse | Databricks Blog

I had no idea BladeBridge existed, so I'm pleased to see it's part of the Databricks offerings soon. Some problems solve themselves.

Re: Redshift Stored Procedure Migration to Databricks

nayan_wylde — Tue, 17 Jun 2025 18:36:40 GMT

Lake Bridge might be a solution.

https://www.databricks.com/solutions/migration/lakebridge

Re: Redshift Stored Procedure Migration to Databricks

mnorland — Tue, 17 Jun 2025 23:22:30 GMT

Talk to your Databricks account rep about whether there is a preview still open on Stored Procedures.

Re: Redshift Stored Procedure Migration to Databricks

jerrygen78 — Wed, 18 Jun 2025 05:18:13 GMT

Hi Avinash, You could give ChatGPT a shot — just ask it to help convert it for you. You might honestly be surprised by how well it handles it. There are more advanced ways to do it too, though those can get a bit tricky if you're not used to them.

Re: Redshift Stored Procedure Migration to Databricks

nayan_wylde — Wed, 18 Jun 2025 13:05:23 GMT

Databricks docs shows procedures are in public preview and requires runtime 17.0 and above.

https://docs.databricks.com/aws/en/sql/language-manual/sql-ref-syntax-ddl-create-procedure