Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
Is it possible to direct spark.sparkContext.setCheckpointDir to an external Azure Storage Container location (instead of DBFS) & if so how, there's very little documentation on that.