Getting errors while reading data from URL

[sharma_kamal]

I'm encountering some issues while trying to read a public dataset from a URL using Databricks. Here's the code snippet(along with errors) I'm working with:

I'm confused about Delta format error here.

• When I read data from a URL, how would it have a Delta log associated with it? Delta logs seem relevant for data stored in Databricks, not external URLs.
• Why is Databricks suggesting Delta format for this scenario?

I'm a bit stuck here. Any pointers or advice, please?

[MuthuLakshmi · Accepted solution]

@sharma_kamal 

Please disable the formatCheck in notebook and check if you could read the data

The configuration command %sql SET spark.databricks.delta.formatCheck.enabled=false will disable the format check for Delta tables in Databricks.

Databricks delta format check is a feature that validates whether the format of the source files is one of CSV, JSON, AVRO, ORC, PARQUET, TEXT, or BINARYFILE and if that meets that condition it will raise exception that you are facing.

This format check is because to ensure only delta format must be loaded for any operation.

If the format check is disabled, Databricks will not perform this validation and bypass this format check.

[sharma_kamal]

Thank you @MuthuLakshmi . It helped.