I am trying to load a delta table from a dataframe. But its giving me an error.

RahulMukherjee — Tue, 13 Aug 2019 15:14:50 GMT

Code :

from pyspark.sql.functions import *

acDF = spark.read.format('csv').options(header='true', inferschema='true').load("/mnt/rahulmnt/Insurance_Info1.csv");

acDF.write.option("overwriteSchema", "true").format("delta").mode("overwrite").save("/delta/InsuranceInput")

Error :

AnalysisException: 'Incompatible format detected.\n\nYou are trying to write to

/delta/InsuranceInput

using Databricks Delta, but there is no\ntransaction log present. Check the upstream job to make sure that it is writing\nusing format("delta") and that you are trying to write to the table base path.\n\nTo disable this check, SET spark.databricks.delta.formatCheck.enabled=false\nTo learn more about Delta, see https://docs.azuredatabricks.net/delta/index.html\n;'

Re: I am trying to load a delta table from a dataframe. But its giving me an error.

AbhaKhanna — Fri, 28 Aug 2020 08:40:46 GMT

1. using Spark SQL Context in python,

scala

notebooks :

sql("SET spark.databricks.delta.formatCheck.enabled=false")

2. In

SQL dbc notebooks

SET spark.databricks.delta.formatCheck.enabled=false

topic I am trying to load a delta table from a dataframe. But its giving me an error. in Data Engineering

I am trying to load a delta table from a dataframe. But its giving me an error.

Re: I am trying to load a delta table from a dataframe. But its giving me an error.