topic Re: Are Python External UDFs supported in Databricks SQL warehouse? in Data Engineering

Are Python External UDFs supported in Databricks SQL warehouse?

AnubhavG — Sat, 12 Nov 2022 07:44:06 GMT

I tried running a python UDF in the Databricks SQL warehouse but it did not run and gave the "Python UDF is not supported" error.

Can i get a clear picture if the Python External UDFs are supported or not?

Re: Are Python External UDFs supported in Databricks SQL warehouse?

PriyaAnanthram — Mon, 14 Nov 2022 00:15:14 GMT

Isnt this in private prview currently https://www.databricks.com/blog/2022/07/22/power-to-the-sql-people-introducing-python-udfs-in-databricks-sql.html

Re: Are Python External UDFs supported in Databricks SQL warehouse?

AnubhavG — Mon, 14 Nov 2022 00:53:59 GMT

Isn't it released for the public preview yet?

In a webinar, I heard it is planned for Q3.

Re: Are Python External UDFs supported in Databricks SQL warehouse?

LandanG — Mon, 14 Nov 2022 14:42:43 GMT

Hi @Anubhav Gupta ,

UDF's are currently still in private preview with the public preview being pushed back a bit. You can ask your Databricks rep/SA to see if you're eligible (due to early stage technical requirements) for the preview

Re: Are Python External UDFs supported in Databricks SQL warehouse?

AnubhavG — Tue, 15 Nov 2022 04:49:01 GMT

Thanks for the reply @Landan George

Also Do we have the capability of calling an API using Python UDF in private preview?

If not then ,Is there any way we can call an API in Databricks SQL by any chance?

Re: Are Python External UDFs supported in Databricks SQL warehouse?

youssefmrini — Tue, 15 Nov 2022 14:25:45 GMT

It's a private preview and It will be supported only in PRO SQL Warehouse and the Serverless SQL Warehouse

Re: Are Python External UDFs supported in Databricks SQL warehouse?

LandanG — Tue, 15 Nov 2022 16:28:25 GMT

@Anubhav Gupta As of right now there is no network connectivity for SQL UDFs (so no API calls) but that may come later down the line.

I'm not sure if there's a way to use APIs in DB SQL but depending on your use case there might be a workaround using different features.

Re: Are Python External UDFs supported in Databricks SQL warehouse?

AnubhavG — Tue, 15 Nov 2022 17:58:24 GMT

@Landan George The use case is I want to tokenize/encrypt the SQL data using the API call.

Is there a way I can achieve this.

Re: Are Python External UDFs supported in Databricks SQL warehouse?

Ajay-Pandey — Tue, 22 Nov 2022 12:54:29 GMT

Thanks