Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
10-15-2024 01:43 AM
My code :
import datetime
from pyspark.sql.functions import lit
now=datetime.datetime.now()
tempTableName=f"xxx.xxx.xxxx";
stageDf = spark.read.format("json").load('https://xxxx.blob.core.xxxx.xx/insights-activity-logs/xxxx/PT1H.json')
stageDf=stageDf.withColumn("LastUpdateTime_",lit(now))
stageDf.write.format("delta").mode("overwrite").saveAsTable(tempTableName)Error msg:
[DELTA_INVALID_FORMAT] Incompatible format detected.