topic Re: Does Databricks encrypt Spark's spilled data? in Data Engineering https://community.databricks.com/t5/data-engineering/does-databricks-encrypt-spark-s-spilled-data/m-p/27083#M18983 <P>the spilled data is written to some object store on the cloud provider.</P><P>I believe all of them apply encryption by default.</P><P>Of course it is up to you (or your colleagues) to restrict access to the storage.​</P> Fri, 25 Feb 2022 07:16:15 GMT -werners- 2022-02-25T07:16:15Z Does Databricks encrypt Spark's spilled data? https://community.databricks.com/t5/data-engineering/does-databricks-encrypt-spark-s-spilled-data/m-p/27082#M18982 Thu, 24 Feb 2022 18:18:52 GMT https://community.databricks.com/t5/data-engineering/does-databricks-encrypt-spark-s-spilled-data/m-p/27082#M18982 Atacama 2022-02-24T18:18:52Z Re: Does Databricks encrypt Spark's spilled data? https://community.databricks.com/t5/data-engineering/does-databricks-encrypt-spark-s-spilled-data/m-p/27083#M18983 <P>the spilled data is written to some object store on the cloud provider.</P><P>I believe all of them apply encryption by default.</P><P>Of course it is up to you (or your colleagues) to restrict access to the storage.​</P> Fri, 25 Feb 2022 07:16:15 GMT https://community.databricks.com/t5/data-engineering/does-databricks-encrypt-spark-s-spilled-data/m-p/27083#M18983 -werners- 2022-02-25T07:16:15Z Re: Does Databricks encrypt Spark's spilled data? https://community.databricks.com/t5/data-engineering/does-databricks-encrypt-spark-s-spilled-data/m-p/27084#M18984 <P>Let's say we're talking about Azure. As far as I know each VM has its own disk. </P> Fri, 25 Feb 2022 16:13:23 GMT https://community.databricks.com/t5/data-engineering/does-databricks-encrypt-spark-s-spilled-data/m-p/27084#M18984 Atacama 2022-02-25T16:13:23Z Re: Does Databricks encrypt Spark's spilled data? https://community.databricks.com/t5/data-engineering/does-databricks-encrypt-spark-s-spilled-data/m-p/27085#M18985 <P>OK get it.</P><P>You are looking for local disk encryption.</P><P>For Azure you can <A href="https://docs.microsoft.com/en-us/azure/databricks/clusters/configure#local-disk-encryption" alt="https://docs.microsoft.com/en-us/azure/databricks/clusters/configure#local-disk-encryption" target="_blank">find the docs here</A>. Mind that this is in public preview so not enabled by default.</P><P>There are also other encryption options, which <A href="https://docs.microsoft.com/en-us/azure/databricks/security/" alt="https://docs.microsoft.com/en-us/azure/databricks/security/" target="_blank">you can find here</A>.</P> Sat, 26 Feb 2022 07:33:26 GMT https://community.databricks.com/t5/data-engineering/does-databricks-encrypt-spark-s-spilled-data/m-p/27085#M18985 -werners- 2022-02-26T07:33:26Z