I have a column that contains an array of structs as follows:

"column" : [ 
{ "struct_field1": "struct_value",  "struct_field2": "struct_value" }, 
{ "struct_field1": "struct_value",  "struct_field2": "struct_value" } 
]

I want to apply a udf to each field of the structs. I am currently trying to do this using a transform however does not seem to work because the udf is not receiving the context.

The error I get is "Cannot generate code for expression: <lambda>(lambda x_1#123.struct_field1)#45678"

df.select(transform("column", lambda x: struct( 
  my_udf_for(x.struct_field1).alias("struct_field1"),
  my_udf_for(x.struct_field2).alias("struct_field2"),
)).alias("column"))

How do I nest a udf inside a transform?