Delete a file from GCS folder
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
11-24-2022 03:09 PM
What is the best way to delete files from the gcp bucket inside spark job?
Labels:
- Labels:
-
Delete File
-
Gcs
1 REPLY 1
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
11-24-2022 08:06 PM
@M Baig
yes you need just to create service account for databricks and than assign storage admin role to bucket. After that you can mount GCS standard way:
- bucket_name = "<bucket-name>"
- mount_name = "<mount-name>"
- dbutils.fs.mount("gs://%s" % bucket_name, "/mnt/%s" % mount_name)
- display(dbutils.fs.ls("/mnt/%s" % mount_name))
here link for ur refer before process, hope some info help on this case
https://docs.gcp.databricks.com/external-data/gcs.html
cheer

