@thomasthomas 
What I would do is using the RESTORE function to rollback in case of a failure.
It would work like this:

from pyspark.sql.functions import max as _max, col

tgt_table_name = "catalog.schema.tbl_name"

# Get current table version
ver_df = (
   spark.sql(f"DESCRIBE HISTORY {tgt_table_name}")
        .select(_max(col("version")).alias("version"))
)

tbl_ver = df.collect()[0].version

try:
   # Your code to transfer data here

except Exception:
   spark.sql(f"RESTORE TABLE {tgt_table_name} TO VERSION AS OF {tbl_ver}")
   raise Exception(f"Load of {tgt_table_name} failed. Restored to {tbl_ver}")