[UDF_MAX_COUNT_EXCEEDED] Exceeded query-wide UDF limit of 5 UDFs

marianopenn
New Contributor III

We are using DLT to ingest data into our Unity catalog and then, in a separate job, we are reading and manipulating this data and then writing it to a table like:

df.write.saveAsTable(name=target_table_path)


We are getting an error which I cannot find ANY documentation about:

SparkRuntimeException: [UDF_MAX_COUNT_EXCEEDED] Exceeded query-wide UDF limit of 5 UDFs (limited during public preview). Found 14. 

What does this mean and how can we address it?
Running on an i3.xlarge shared cluster, DBR 13.2, Spark 3.4.0, Scala 2.12.