cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
cancel
Showing results for 
Search instead for 
Did you mean: 

pyspark.sql.utils.AnalysisException: Non-time-based windows are not supported on streaming DataFrames/Datasets

Gaurav_784295
New Contributor III

pyspark.sql.utils.AnalysisException: Non-time-based windows are not supported on streaming DataFrames/Datasets

Getting this error while writing can any one please tell how we can resolve it

2 REPLIES 2

Aviral-Bhardwaj
Esteemed Contributor III

Please give us the whole code by that we can understand what exactly you are doing

or you can connect with me in my LinkedIN and we can setup meeting.My LinkedIn URL is in my profile

Gaurav_784295
New Contributor III

I'm trying to run query on some table and then storing that result in some table .

query = stream

.writeStream

.format("delta")

.foreachBatch(batch_function) \

.option('checkpointLocation', self.checkpoint_loc)

.trigger(processingTime=self.trigger_interval)

And in that query I'm first using union function and then using window function. Their I'm facing this error pyspark.sql.utils.AnalysisException: Non-time-based windows are not supported on streaming DataFrames/Datasets

Welcome to Databricks Community: Lets learn, network and celebrate together

Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections. 

Click here to register and join today! 

Engage in exciting technical discussions, join a group with your peers and meet our Featured Members.