how to pass arguments and variables to databricks python activity from azure data factory

[MallikSunkara]

how to pass arguments and variables to databricks python activity from azure data factory

[Anonymous]

I do something like this...

In DBrix notebook:

dbutils.widgets.text("runDateYYYYMMDD", "")
runDate = dbutils.widgets.get("runDateYYYYMMDD")

In Data Factory:

Pipeline > Parameters tab > New Parameter:

Name: RunDate, Type:String, Default Value:

Data Bricks Notebook Activity > Settings tab > New Base Parameter:

Name=runDateYYYYMMDD, Value=pipeline().parameters.RunDate

[MallikSunkara]

Hi Timkracht,

thanks for your reply.

is there any way to directly to read those parameters without using widgets. what we need is from azure data factory we are triggering python activity as python program file is stored in dbfs, now i need to access those variables in that python program.

what we need is which python files are being executed by which pipeline and activity of azure data factory.

thanks in advance.

[Anonymous]

I'm afraid I do not have experience with that, just passing parameters through widgets in notebooks.

[CristianIspan]

try importing argv from sys. Then if you have the parameter added correctly in DataFactory you could get it in your python script typing argv[1] (index 0 is the file path).