cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

how to pass arguments and variables to databricks python activity from azure data factory

MallikSunkara
New Contributor II

how to pass arguments and variables to databricks python activity from azure data factory

4 REPLIES 4

Anonymous
Not applicable

I do something like this...

In DBrix notebook:

dbutils.widgets.text("runDateYYYYMMDD", "")
runDate = dbutils.widgets.get("runDateYYYYMMDD")

In Data Factory:

Pipeline > Parameters tab > New Parameter:

Name: RunDate, Type:String, Default Value:

Data Bricks Notebook Activity > Settings tab > New Base Parameter:

Name=runDateYYYYMMDD, Value=pipeline().parameters.RunDate

Hi Timkracht,

thanks for your reply.

is there any way to directly to read those parameters without using widgets. what we need is from azure data factory we are triggering python activity as python program file is stored in dbfs, now i need to access those variables in that python program.

what we need is which python files are being executed by which pipeline and activity of azure data factory.

thanks in advance.

Anonymous
Not applicable

I'm afraid I do not have experience with that, just passing parameters through widgets in notebooks.

CristianIspan
New Contributor II

try importing argv from sys. Then if you have the parameter added correctly in DataFactory you could get it in your python script typing argv[1] (index 0 is the file path).

Join 100K+ Data Experts: Register Now & Grow with Us!

Excited to expand your horizons with us? Click here to Register and begin your journey to success!

Already a member? Login and join your local regional user group! If there isn’t one near you, fill out this form and we’ll create one for you to join!