How to change compression codec of sql warehouse written files?
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-02-2023 07:39 AM
Hi, I'm currently starting to use SQL Warehouse, and we have most of our lake in a compression different than snappy.
How can I set the SQL warehouse to use a compression like gzip, zstd, on CREATE, INSERT, etc?
Tried this:
set spark.sql.parquet.compression.codec=gzip
And got the error:
Error running query: [_LEGACY_ERROR_TEMP_DBR_0222] org.apache.spark.sql.AnalysisException: Configuration spark.sql.parquet.compression.codec is not available.
Thanks!
Labels:
- Labels:
-
Parquet
-
SQL
-
Sql Warehouse
1 REPLY 1
Anonymous
Not applicable
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-15-2023 11:55 PM
Hi @Alejandro Martinez
Great to meet you, and thanks for your question!
Let's see if your peers in the community have an answer to your question. Thanks.