Further to this, it also seems that it is not possible to set a checkpoint directory on an external location where the principal has write permission to the external location. When we try:spark.sparkContext.setCheckpointDir("s3://bucket/path")we see:...
