https://community.databricks.com/t5/data-engineering/how-to-remove-more-than-4-byte-characters-using-pyspark-in/m-p/27809#M19657 <P>assuming you are having a string type column in pyspark dataframe, one possible way could be</P><OL><LI>identify total number of characters for each value in column (say </LI><LI>identify no of bytes taken by each character (say b)</LI><LI>use substring() function to select first n characters where n = floor(4 / b)</LI></OL> Tue, 29 Nov 2022 11:00:30 GMT Shalabh007 2022-11-29T11:00:30Z https://community.databricks.com/t5/data-engineering/how-to-remove-more-than-4-byte-characters-using-pyspark-in/m-p/27808#M19656 <P>Hi community,</P><P></P><P>We have the need of removing more than 4 byte characters using pyspark in databricks since these are not supported by amazon Redshift. Does someone know how can I accomplish this?</P><P></P><P>Thank you very much in advance</P><P></P><P>Regards</P> Thu, 17 Feb 2022 04:49:28 GMT https://community.databricks.com/t5/data-engineering/how-to-remove-more-than-4-byte-characters-using-pyspark-in/m-p/27808#M19656 eimis_pacheco 2022-02-17T04:49:28Z https://community.databricks.com/t5/data-engineering/how-to-remove-more-than-4-byte-characters-using-pyspark-in/m-p/27809#M19657 <P>assuming you are having a string type column in pyspark dataframe, one possible way could be</P><OL><LI>identify total number of characters for each value in column (say </LI><LI>identify no of bytes taken by each character (say b)</LI><LI>use substring() function to select first n characters where n = floor(4 / b)</LI></OL> Tue, 29 Nov 2022 11:00:30 GMT https://community.databricks.com/t5/data-engineering/how-to-remove-more-than-4-byte-characters-using-pyspark-in/m-p/27809#M19657 Shalabh007 2022-11-29T11:00:30Z