Delete a file from GCS folder

MBV3
Contributor

What is the best way to delete files from the gcp bucket inside spark job?

Unforgiven
Valued Contributor III

@M Baig​ 

yes you need just to create service account for databricks and than assign storage admin role to bucket. After that you can mount GCS standard way:

  1. bucket_name = "<bucket-name>"
  2. mount_name = "<mount-name>"
  3. dbutils.fs.mount("gs://%s" % bucket_name, "/mnt/%s" % mount_name)
  4. display(dbutils.fs.ls("/mnt/%s" % mount_name))

here link for ur refer before process, hope some info help on this case

https://docs.gcp.databricks.com/external-data/gcs.html

cheer