Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
I am creating Databricks jobs and tasks using a python package. I have defined a task level parameter and would like to reference it in my script using sys.argv. How can I do that ?
Thanks, but the link works for notebook. I have a python package run as python wheel and am wondering how to access the parameters. When I run the job its not able to understand the task level parameters in sys.argv
Join Us as a Local Community Builder!
Passionate about hosting events and connecting people? Help us grow a vibrant local communityโsign up today to get started!