Hi Phani, what exactly are you looking for with best practices?

At a high level:

• Always provide an external storage location (S3, ADLS, GCS) for your pipeline
• Use Auto Scaling!
• Python imports can be leverage to reuse code

With regards to providing a storage location, if you put all your pipelines in a common storage location it makes it easier to read all the associated event logs for pipeline monitoring as well.