Databricks Community

There may be more information in the log. Look for "Cannot parse the zip archive."

It seems like a permission error.  Can you check if the managed identity has correct permissions to write to the volume?DLT should be supported to write into volumes.

Hey @param_sen Given your concerns about expensive joins and prioritizing analytics with flat raw data, here are some suggestions:Analyze the most common queries and reports you anticipate. Do they heavily rely on joins across dimensions? If not, the...

it might be a bug which is already logged, or a new one.  You can check the Spark Jira pages.

@Hardy I think you are using the word volume in the path but it should be Volumes(plural), not Volume(singular). Do one thing, copy the volume path directly from the Workspace and try.

Hi, sorry I guess my response wasn't sent. The source are JSON files that are uploaded to an S3 bucket. The sink will be a Delta Table and we are using autoloader.The question was about the compression format of the incoming JSON files, e.g. if it wo...

Hi @FerArribas  The first digits of the jobgroup that are shown in the spark UI are execContextId and cmdId(Command_ID). You can think of the execContextId as some kind of “REPL ID” For example, if you take the below job group ID as an example, jobGr...

Hi,Hopefully this question is related to testing and any production data would get persisted to a table but one example is:df = (spark.range(10).write.format("delta").mode("append").save("file:/tmp/data"))ALTER TABLE delta.`file:/tmp/data` CLUSTER BY...