Redshift Stored Procedure Migration to Databricks

[Avinash_Narala]

Hi,

I want to migrate Redshift SQL Stored Procedures to databricks.

As databricks doesn't support the concept of SQL Stored Procedures. 

How can I do so?

[MariuszK]

You can create a Python function or move logic a notebook, it depend what is the purpose of the procedure.

[Avinash_Narala]

i have a bunch of procedures to convert. so it is not possible to generalise the purpose of each procedure. 

I need a wrapper program to convert redshift sql procedures into python functions in databricks notebooks

[Rjdudley]

I am assuming you want a sproc so you have some code which is reusable across many notebooks.  If so, Databricks supports several types of functions: https://learn.microsoft.com/en-us/azure/databricks/sql/language-manual/sql-ref-functions

However if your Redshift sprocs are all of the processing logic, you should convert those to notebooks instead.

I don't know of any program which will directly convert Redshift SQL to Python (or Databricks SQL).  An LLM might help, perhaps even the Databricks Assistant in the cloud IDE.

[MariuszK]

You can try with ChatGPT, ask the chat to convert it, you will be really suppressed. There is more advance option for that, but it's more complex.

[Rjdudley]

Hey @Avinash_Narala in case you didn't see this announcement, looks like it might solver your problem: Welcoming BladeBridge to Databricks: Accelerating Data Warehouse Migrations to Lakehouse | Databrick...

I had no idea BladeBridge existed, so I'm pleased to see it's part of the Databricks offerings soon.  Some problems solve themselves.

[nayan_wylde]

Lake Bridge might be a solution.

https://www.databricks.com/solutions/migration/lakebridge

[mnorland]

Talk to your Databricks account rep about whether there is a preview still open on Stored Procedures.

[jerrygen78]

Hi Avinash, You could give ChatGPT a shot — just ask it to help convert it for you. You might honestly be surprised by how well it handles it. There are more advanced ways to do it too, though those can get a bit tricky if you're not used to them.

[nayan_wylde]

Databricks docs shows procedures are in public preview and requires runtime 17.0 and above.

https://docs.databricks.com/aws/en/sql/language-manual/sql-ref-syntax-ddl-create-procedure