cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

Column is accessible after dropping the same column

Ajay-Pandey
Esteemed Contributor III

Hi

Today I have seen very Strang behavior of databricks.

I have dropped one column from a dataframe and assigned the result to a new dataframe but I am able to use the dropped column in the filter command.

In general scenario I should get an error but it's running fine.

Attaching snapshot for the same

image.png@Ben Hayesโ€‹ 

 @Entong Shenโ€‹ 

@Hubert Dudekโ€‹ 

#DAIS2023โ€‹ 

Ajay Kumar Pandey
1 ACCEPTED SOLUTION

Accepted Solutions

Sandeep
Contributor III

@Ajay Pandeyโ€‹ , this is a known behavior. Please refer this JIRA for details: https://issues.apache.org/jira/browse/SPARK-30421

View solution in original post

2 REPLIES 2

Sandeep
Contributor III

@Ajay Pandeyโ€‹ , this is a known behavior. Please refer this JIRA for details: https://issues.apache.org/jira/browse/SPARK-30421

Ajay-Pandey
Esteemed Contributor III

Thanks

Ajay Kumar Pandey

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโ€™t want to miss the chance to attend and share knowledge.

If there isnโ€™t a group near you, start one and help create a community that brings people together.

Request a New Group