Databricks Community

Giri-Patcham · 05-28-2025

Summary Understand Spark's StateStore API: Learn how this core component enables robust stateful operations in Structured Streaming through versioned, key-value state management and checkpointing for fault tolerance.Leverage RocksDB for Performance: ...

Giri-Patcham · 07-10-2024

@pavansharma36 Try something like this PUT <DATABRICKS_HOST>/api/2.0/fs/directories/Volumes/samples/default/test

Giri-Patcham · 07-08-2024

@pavansharma36 You can try volumes and use Files API to upload https://docs.databricks.com/api/azure/workspace/files

Giri-Patcham · 07-02-2024

@Wolverine Try %fs rm -r sampledir to delete recursively

Giri-Patcham · 07-02-2024

@ajithgaade can you try setting this conf spark.conf.set("spark.databricks.delta.schema.autoMerge.enabled", "true")

Giri-Patcham · 07-02-2024

@alesventus I think "Metastore is down" here is related to the legacy Hive metastore. You can find the stack trace in the driver logs. In your case do you need Hivemetastore?

Databricks Community

User Stats

User Activity

Demystifying Spark's StateStore API for Robust Stateful Streaming

Re: Databricks Workspace import api size limitation

Re: Databricks Workspace import api size limitation

Re: Databricks Magic Command

Re: Autoloader includeExistingFiles with retry didn't update the schema

Re: Unity Catalog metastore is down error