Facing issue when doing merge of dataframe to delta table which has mask applied on two of the columns.

Code 

DeltaTable.forName(sparkSession=spark,tableOrViewName=f'{catalog}.{schema}.{table_name}').alias('target').merge(
    new_df.alias('updates'),
    'updates.customerID = target.customerID'
).whenNotMatchedInsertAll().execute()

Error

[MISSING_ATTRIBUTES.RESOLVED_ATTRIBUTE_APPEAR_IN_OPERATION] Resolved attribute(s) "first_name" missing from "customerID", "first_name", "last_name", "city", "state" in operator !Project [customerID#153L, redact AS first_name#178, redact AS last_name#179, redact AS city#180, redact AS state#181]. Attribute(s) with the same name appear in the operation: "first_name".
Please check if the right attribute(s) are used. SQLSTATE: XX000
File <command-6480109877101749>, line 4
      1 DeltaTable.forName(sparkSession=spark,tableOrViewName=f'{catalog}.{schema}.{table_name}').alias('target').merge(
      2     new_df.alias('updates'),
      3     'updates.customerID = target.customerID'
----> 4 ).whenNotMatchedInsertAll().execute()
File /databricks/spark/python/pyspark/sql/connect/client/core.py:2377, in SparkConnectClient._handle_rpc_error(self, rpc_error)
   2363                 raise SparkConnectGrpcException(
   2364                     "Python versions in the Spark Connect client and server are different. "
   2365                     "To execute user-defined functions, client and server should have the "
   (...)
   2373                         "sqlState", default=SparkConnectGrpcException.CLIENT_UNEXPECTED_MISSING_SQL_STATE),
   2374                 ) from None
   2375             # END-EDGE
-> 2377             raise convert_exception(
   2378                 info,
   2379                 status.message,
   2380                 self._fetch_enriched_error(info),
   2381                 self._display_server_stack_trace(),
   2382             ) from None
   2384     raise SparkConnectGrpcException(
   2385         message=status.message,
   2386         sql_state=SparkConnectGrpcException.CLIENT_UNEXPECTED_MISSING_SQL_STATE,  # EDGE
   2387     ) from None
   2388 else:

Merge works fine with spark.sql or with %sql but have issue with python syntax.

If the column mask is removed from the table, it works fine.