https://community.databricks.com/t5/get-started-discussions/read-zstd-file-from-databricks/m-p/49447#M1601 <P>Thanks for clarifying!</P> Wed, 18 Oct 2023 07:36:20 GMT bigt23 2023-10-18T07:36:20Z https://community.databricks.com/t5/get-started-discussions/read-zstd-file-from-databricks/m-p/49435#M1597 <P>I just started to read `zstd` compressed file in Databricks on Azure, Runtime 14.1 on Spark 3.5.0</P><P>I've set PySpark commands as follows</P><LI-CODE lang="markup">path = f"wasbs://{container}@{storageaccount}.blob.core.windows.net/test-zstd" schema = "some schema" df = spark.read.option("compression", "zstd").json(path, schema) df.createOrReplaceTempView("TestTable")</LI-CODE><P>then hit sql</P><LI-CODE lang="markup">%sql select * from TestTable limit 100</LI-CODE><P>but failed with the following error.</P><LI-CODE lang="markup">Error in SQL statement: IllegalArgumentException: Codec [zstd] is not available. Known codecs are bzip2, deflate, uncompressed, lz4, gzip, snappy, none.</LI-CODE><P>&nbsp;Is there anyway to read `zstd` files there?</P> Wed, 18 Oct 2023 04:32:32 GMT https://community.databricks.com/t5/get-started-discussions/read-zstd-file-from-databricks/m-p/49435#M1597 bigt23 2023-10-18T04:32:32Z https://community.databricks.com/t5/get-started-discussions/read-zstd-file-from-databricks/m-p/49440#M1598 <P>The available compression types are format dependent.<BR />For json, zstd is not (yet) available, whereas for parquet it is.</P> Wed, 18 Oct 2023 05:44:25 GMT https://community.databricks.com/t5/get-started-discussions/read-zstd-file-from-databricks/m-p/49440#M1598 -werners- 2023-10-18T05:44:25Z https://community.databricks.com/t5/get-started-discussions/read-zstd-file-from-databricks/m-p/49447#M1601 <P>Thanks for clarifying!</P> Wed, 18 Oct 2023 07:36:20 GMT https://community.databricks.com/t5/get-started-discussions/read-zstd-file-from-databricks/m-p/49447#M1601 bigt23 2023-10-18T07:36:20Z