cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Sample streaming table is failed

shsalami
New Contributor III
  • Running the following databricks sample code in the pipeline:
 
CREATE OR REFRESH STREAMING TABLE customers
AS SELECT * FROM cloud_files("/databricks-datasets/retail-org/customers/", "csv")
 
  • I got error:
org.apache.spark.sql.catalyst.ExtendedAnalysisException: Unable to process statement for Table 'customers'.
com.databricks.sql.cloudfiles.errors.CloudFilesIllegalArgumentException: Reading from a Delta table is not supported with this syntax. If you would like to consume data from Delta, please refer to the docs: read a Delta table (https://docs.microsoft.com/azure/databricks/delta/tutorial#read), or read a Delta table as a stream source (https://docs.microsoft.com/azure/databricks/structured-streaming/delta-lake#table-streaming-reads-an...). The streaming source from Delta is already optimized for incremental consumption of data.
2 REPLIES 2

filipniziol
New Contributor III

Hi @shsalami ,

The error you're encountering suggests you are attempting to read a Delta table as if it were a collection of CSV files using the cloud_files function.
You may run "DESCRIBE DETAIL customers" and check whether this table exists and it is located in "/databricks-datasets/retail-org/customers/". The other way is to navigate to this location in your cloud storage. It may happen that you already created a Delta table in that location before. The syntax for reading from delta table is different.

 

shsalami
New Contributor III

There is no table with that name.

Also, in that folder just the following file exists:

dbfs:/databricks-datasets/retail-org/customers/customers.csv

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group