Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
files in dbfs are typically not linked to a cluster or it's lifetime. There are tmp directories in dbfs so perhaps you are looking at those, but f.e. Filestore can definitely be used. However, I suggest not using dbfs but some data lake (S3/ADLS).
Join Us as a Local Community Builder!
Passionate about hosting events and connecting people? Help us grow a vibrant local communityโsign up today to get started!
