How to unload a Jar for UDF without restart spark context?

[vonjack]

In the scala notebook of databricks, I created a temporary function with a certain Jar and class name. 

Then I want to update the Jar. But without restart the context, I can not reload the new Jar, the temporary function always reuses the old classes.

I noticed that for normal function it support refresh, but temporary function doesn't.

Does anyone know the way to resolve this issue?

[vonjack]

Hi, I tried updated Jar, drop and re-create the temporary function, the UDF still use the old Jar.

I don't want to restart context and cleanup workspace, are there any other better solutions?

[vonjack]

So if I don't want to restart the cluster, do you think the versionized jar name and class name would be a better solution?

At first, I used this command to create the original UDF:

create or replace temporary function test_udf as 'com.example.TestUDF_v1' using jar '/tmp/test_udf_v1.jar';

Then I want to update the UDF, I'll try that: 

create or replace temporary function test_udf as 'com.example.TestUDF_v2' using jar '/tmp/test_udf_v2.jar';

But that may be into another problem, if different jars have the same class name like 'com.example.SameClass', which version of class will spark cluster use eventually without restart?