@Tim zhang​ :

The issue is that the formatAccount function is defined as a Scala function, but SparkSQL is looking for a SQL function. You need to register the Scala function as a SQL function so that it can be called from SparkSQL. You can register the Scala function as a SQL function using the spark.udf.register method. Here is an example code snippet:

import org.apache.spark.sql.functions.udf
 
val formatAccountUDF = udf((account: String, mask:String ) => {
  val formatter = new MaskFormatter(mask.replace("X", "A"))
  formatter.setValueContainsLiteralCharacters(false)
  val formatAccount = formatter.valueToString(account)
  formatAccount
})
 
spark.udf.register("formatAccount", formatAccountUDF)

After registering the function, you can use it in your SparkSQL queries:

sql("""select formatAccount("1222233334", "X-XXXX-XXXX-X")""")

Hopefully this should work.