Apologies my above comment on the datetime format is wrong of course, the format in the article is correct, I just forgot to add the legacy timeParserPolicy settings. #To Parse Timestamps correctlyspark.conf.set("spark.sql.legacy.timeParserPolicy", "...
@silvyAlej You need to write python code to read the model.json file.You can view the databricks code here .Alternatively you can copy the below and change for your needs:def get_spark_type(data_type, max_length, traits=None):if data_type == 'guid':r...
This is a great article thank you. However please correct the timestamp in the code snippet, it should be : cdm_timestamp_format = "MM/dd/yyyy hh:mm:ss a"