Column is accessible after dropping the same column

Ajay-Pandey
Databricks MVP

Hi

Today I have seen very Strang behavior of databricks.

I have dropped one column from a dataframe and assigned the result to a new dataframe but I am able to use the dropped column in the filter command.

In general scenario I should get an error but it's running fine.

Attaching snapshot for the same

image.png@Ben Hayes​ 

 @Entong Shen​ 

@Hubert Dudek​ 

#DAIS2023​ 

Ajay Kumar Pandey

Sandeep
Databricks Employee
Databricks Employee

@Ajay Pandey​ , this is a known behavior. Please refer this JIRA for details: https://issues.apache.org/jira/browse/SPARK-30421

View solution in original post

Thanks

Ajay Kumar Pandey