https://community.databricks.com/t5/data-engineering/how-to-translate-apache-pig-filter-statement-to-spark/m-p/22720#M15611 <P>If you have the following Apache Pig FILTER statement:</P><PRE><CODE>XCOCD_ACT_Y = FILTER XCOCD BY act_ind == 'Y';</CODE></PRE><P>the equivalent code in Apache Spark is:</P><PRE><CODE>XCOCD_ACT_Y_DF = (XCOCD_DF .filter(col("act_ind") == "Y"))</CODE></PRE><P></P> Fri, 18 Jun 2021 16:52:47 GMT User15787040559 2021-06-18T16:52:47Z https://community.databricks.com/t5/data-engineering/how-to-translate-apache-pig-filter-statement-to-spark/m-p/22720#M15611 <P>If you have the following Apache Pig FILTER statement:</P><PRE><CODE>XCOCD_ACT_Y = FILTER XCOCD BY act_ind == 'Y';</CODE></PRE><P>the equivalent code in Apache Spark is:</P><PRE><CODE>XCOCD_ACT_Y_DF = (XCOCD_DF .filter(col("act_ind") == "Y"))</CODE></PRE><P></P> Fri, 18 Jun 2021 16:52:47 GMT https://community.databricks.com/t5/data-engineering/how-to-translate-apache-pig-filter-statement-to-spark/m-p/22720#M15611 User15787040559 2021-06-18T16:52:47Z