@Mohammad Saber​ :

The error messages you received indicate that the SQL UPDATE command is expecting a column of type map<string,timestamp>, but is instead receiving a string or a non-matching datatype. To resolve this issue, you can convert the Python dictionary to a valid SQL map format using the map_from_entries function in Spark SQL.

Here's an example of how you can use the map_from_entries function to update the table_updates column in your delta table:

from pyspark.sql.functions import map_from_entries
 
# Convert Python dictionary to list of key-value pairs
table_updates_list = list(table_updates_id1.items())
 
# Convert list of key-value pairs to SQL map format
table_updates_map = map_from_entries(table_updates_list)
 
# Use SQL UPDATE command to update delta table
spark.sql(f"""
    UPDATE dev.bronze.test_map
    SET
        table_updates = {table_updates_map}
    WHERE
        id = 1
""")

In this example, the map_from_entries function converts the table_updates_id1 dictionary to a list of key-value pairs, which is then passed to the map_from_entries function to create a SQL map. The resulting SQL map is then used in the SQL UPDATE command to update the table_updates column for rows where id = 1,