extract latest files from ADLS Gen2 mount point in databricks using pyspark
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-01-2021 10:26 AM
Hi Team,
I am trying to get the latest files from an ADLS mount point directory. I am not sure how to extract latest files ,Last modified Date using Pyspark from ADLS Gen2 storage account. Please let me know asap. Thanks!I am looking forward your response.
- Labels:
-
Azure databricks
-
AzureDatalake
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-02-2021 05:16 AM
Hi Team,
can you please respond here. Thanks!
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-10-2021 05:17 AM
Hi Team,
Can someone please respond here. If someone has already implemented. Thanks!
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-02-2022 01:48 AM
I am looking for the same too. Appreciate the speedy response.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-02-2022 07:00 AM
Hi @pankaj92 ,
I wrote a Python code to pick a latest file from mnt location ,
import os
path = "/dbfs/mnt/xxxx"
filelist=[]
for file_item in os.listdir(path):
filelist.append(file_item)
file=len(filelist)
print(filelist[file-1])
Thanks
