11-08-2023 06:19 PM
Hello,
I am trying to generate a DLT but need to use a UDF Table Function in the process.
This is what I have so far, everything works (without e CREATE OR REFRESH LIVE TABLE wrapper)
```sql
```
However, this results in the following error:
```
11-15-2023 03:56 PM
Hi all,
I have figured out my error.
The function creation should NOT be in the same notebook as the one used by the DLT pipeline.
Instead, create the function separately and register it in Unity Catalog, then reference the function in the DLT pipeline.
As a side note, UDF support for DLT is only available in DBR 13.3 and above. At the time of writing this, you will need to configure the runtime to 'preview' over 'current' to enable UDFs in DLT.
11-14-2023 11:34 AM
@alexiswl - could you please use CREATE OR REPLACE function instead of CREATE OR REFRESH LIVE table?
11-14-2023 06:58 PM
Hi @shan_chandra, unfortunately this is in the context of creating a DLT, so if I update as suggested, I get the following error:
```
org.apache.spark.sql.AnalysisException: The following code at '/Repos/services+databricks@umccr.org/data-portal-analytics/delta_live_tables/silver/silver_portal_dlt' is not supported in a DLT pipeline: CREATE OR REPLACE FUNCTION silver.portal.get_workf... Please correct your code and make sure you only have DLT statements. For reference: DLT currently accepts 'CREATE MATERIALIZED VIEW', 'CREATE STREAMING TABLE', 'APPLY CHANGES INTO', and 'SET' statements.
```
11-15-2023 03:56 PM
Hi all,
I have figured out my error.
The function creation should NOT be in the same notebook as the one used by the DLT pipeline.
Instead, create the function separately and register it in Unity Catalog, then reference the function in the DLT pipeline.
As a side note, UDF support for DLT is only available in DBR 13.3 and above. At the time of writing this, you will need to configure the runtime to 'preview' over 'current' to enable UDFs in DLT.
11-21-2023 12:45 AM
Hi @alexiswl , I want to express my gratitude for your effort in selecting the most suitable solution. It's great to hear that your query has been successfully resolved. Thank you for your contribution.
Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.
If there isn’t a group near you, start one and help create a community that brings people together.
Request a New Group