Hi,

Thanks for your reply.

Apologies if I am sending the incorrect information. I am in the process of learning databricks, and adding content to my personal blog.

DBR Version

Cluster Configurations

From the Compute menu, this is the only cluster available in the Databricks Free Edition:

From the Workspace Notebook, I  also tried to attach the General Compute option, though this didn't resolve the issue either:

Log4j File

I have attached the Log4j File, for the time I executed the following query, that errored:

## Incrementally (or stream) data using Auto Loader
(spark.
readStream
.format("cloudFiles")
.option("cloudFiles.format", "csv")
.option("header", "true")
.option("sep", ",")
.option("inferSchema", "true")
.option("cloudFiles.schemaLocation", f"{checkpoint_file_location}")
.load("/Volumes/workspace/python_auto_loader/csv_staging")
.writeStream
.option("checkpointLocation", f"{checkpoint_file_location}")
.trigger(once=True)
.toTable("workspace.python_auto_loader.python_csv_autoloader")
)

If you require any additional information, please message me again.

Thank you for your help.

Giuseppe