Databricks Community

aicd_de · 09-19-2023

Hi AllLooking to get some help. We are on Unity Catalog in Azure. We have a requirement to use Python to write out PNG files (several) via Matplotlib and then drop those into an ADLS2 Bucket. With Unity Catalog, we can easily use dbutils.fs.cp or fs....

aicd_de · 01-04-2023

I have a set of Spark Dataframes that I convert into Temp Views to run Spark SQL with. Then, I delete them after my logic/use is complete. The delete step throws an odd error that I am not sure how to fix. Looking for some tips on fixing it. As a not...

aicd_de · 09-21-2023

Hmm I read something different - someone else had this error because they used a shared cluster - apparently it does not happen on a single user cluster. All those settings are already done and I am a fully admin.

aicd_de · 09-20-2023

I get this error writing to that location:"java.lang.SecurityException: Cannot use com.databricks.backend.daemon.driver.WorkspaceLocalFileSystem - local filesystem access is forbidden"

aicd_de · 01-04-2023

spark.sql("DROP TABLE "+prefix_updates) spark.sql("DROP TABLE "+prefix_main)Fixed it for me.

Databricks Community

User Stats

User Activity

Unity Catalog - Writing to PNG Files to Cluster and then using dbutils.fs.cp to send to Azure ADLS2

Error Using spark.catalog.dropTempView()

Re: Unity Catalog - Writing to PNG Files to Cluster and then using dbutils.fs.cp to send to Azure AD

Re: Unity Catalog - Writing to PNG Files to Cluster and then using dbutils.fs.cp to send to Azure AD

Re: Error Using spark.catalog.dropTempView()