databricks structured streaming external table unity catalog

christian_chong
New Contributor III

Hi everbody,

I am facing a issue with spark structured steaming, with databricks on gcp. 

I use external table type but schema are managed schema. 

The code below in my notebook works very well

But if i add column masking to target table, and rerun the notebook i get an error :

"
[RequestId=xxxx-exxx-xxxx-adf4-86b9b7e82252 ErrorClass=INVALID_PARAMETER_VALUE.INVALID_PARAMETER_VALUE] Input path gs://table overlaps with other external tables or volumes. Conflicting tables/volumes: xxx.xxx.table, xxx.xxx.another_table
"

here is a sample of my code

"
df = spark.readStream.load(f"{bronze_table_path}")

df.writeStream \
.format("delta") \
.option("checkpointLocation", f"{silver_checkpoint}") \
.option("mergeSchema", "true") \
.trigger(availableNow=True) \
.outputMode("append") \
.start(path=f"{silver_table_path}")
"


Thanks you