topic Re: Get last modified date or create date for azure blob container in Data Engineering

Get last modified date or create date for azure blob container

AlaQabaja — Thu, 19 Sep 2019 17:02:46 GMT

Hi Everyone,

I am trying to implement a way in Python to only read files that weren't loaded since the last run of my notebook. The way I am thinking of implementing this is to keep of the last time my notebook has finished in a database table. Next time my job runs, I would like to only load the net new files that were added to azure blob. So, I want to be able to read the create date or last modified date for that file for me to be able to do incremental loading.

My two questions are:

- Am I doing it the right way?

- Is it possible to get create date and/or modified date of an azure blob container using Python?

Re: Get last modified date or create date for azure blob container

shyam_9 — Fri, 20 Sep 2019 06:58:31 GMT

Hi @Ala Qabaja,

For Getting last modified date in azure blob container use below code,

DateTime date =DateTime.Now;using(var filestream =File.OpenRead(@"D:\test.txt")){
               block.UploadFromStream(filestream);
               block.Metadata["lastmodifydate"]= date.ToString();
               block.SetMetadata();}
           block.FetchAttributes();string dateInfo= block.Metadata["lastmodifydate"];

Re: Get last modified date or create date for azure blob container

AlaQabaja — Fri, 20 Sep 2019 22:10:34 GMT

Thanks for your response. I am not sure what your code is and how to implement it. I am using PySpark and not using streams.

Re: Get last modified date or create date for azure blob container

Forum_Admin — Wed, 18 Mar 2020 12:25:37 GMT

Hello! I just wanted to share my point of view on the topic of dating sites. I have been looking for a decent Asian catch-up site for a very long time, in addition to them I found https://hookupsearch.org/asian-hookup-sites/. We definitely recommend this to anyone who is going to date and date women from solid anodized dishes, because the platforms have everything you might need, as well as what you want!