https://community.databricks.com/t5/warehousing-analytics/parquet-encryption-decryption-in-databricks/m-p/92634#M1610 <P>Thank you for your reply!</P><P>As I understand, they don’t support columnar encryption automatically; we need to add some logic manually. Is this related to a performance issue that columnar encryption level would cause? What I mean, do they recommend columnar encryption?</P> Thu, 03 Oct 2024 09:59:41 GMT Aya-Ahmed 2024-10-03T09:59:41Z https://community.databricks.com/t5/warehousing-analytics/parquet-encryption-decryption-in-databricks/m-p/92196#M1600 <P><SPAN>Hi everyone,</SPAN></P><P><SPAN>I'm curious about Databricks' approach to encrypting and decrypting Parquet files.</SPAN><SPAN> Does Databricks adhere to standard encryption/decryption methods for Parquet?</SPAN><SPAN> If not,</SPAN><SPAN> what specific methods or techniques are used?</SPAN></P><P><SPAN>I'd appreciate any insights or recommendations on best practices for securing Parquet data in Databricks.</SPAN></P><P><SPAN>Thanks!</SPAN></P> Sun, 29 Sep 2024 09:09:41 GMT https://community.databricks.com/t5/warehousing-analytics/parquet-encryption-decryption-in-databricks/m-p/92196#M1600 Aya-Ahmed 2024-09-29T09:09:41Z https://community.databricks.com/t5/warehousing-analytics/parquet-encryption-decryption-in-databricks/m-p/92277#M1601 <P>Since Databricks uses Spark, you should be able to use e.g. <A href="https://spark.apache.org/docs/latest/sql-data-sources-parquet.html#columnar-encryption" target="_self">Columnar Encryption</A></P><P>Besides, you can look into <A href="https://www.databricks.com/blog/2020/11/20/enforcing-column-level-encryption-and-avoiding-data-duplication-with-pii.html" target="_self">this</A> and <A href="https://spark.apache.org/docs/latest/api/python/reference/pyspark.sql/api/pyspark.sql.functions.aes_encrypt.html" target="_self">aes-specific functions</A>.</P> Mon, 30 Sep 2024 08:12:36 GMT https://community.databricks.com/t5/warehousing-analytics/parquet-encryption-decryption-in-databricks/m-p/92277#M1601 Witold 2024-09-30T08:12:36Z https://community.databricks.com/t5/warehousing-analytics/parquet-encryption-decryption-in-databricks/m-p/92634#M1610 <P>Thank you for your reply!</P><P>As I understand, they don’t support columnar encryption automatically; we need to add some logic manually. Is this related to a performance issue that columnar encryption level would cause? What I mean, do they recommend columnar encryption?</P> Thu, 03 Oct 2024 09:59:41 GMT https://community.databricks.com/t5/warehousing-analytics/parquet-encryption-decryption-in-databricks/m-p/92634#M1610 Aya-Ahmed 2024-10-03T09:59:41Z