- 22845 Views
- 9 replies
- 9 kudos
Resolved! How to specify entry_point for python_wheel_task?
Can someone provide me an example for a python_wheel_task and what the entry_point field should be?The jobs UI help popup says this about "entry_point":"Function to call when starting the wheel, for example: main. If the entry point does not exist in...
- 22845 Views
- 9 replies
- 9 kudos
- 9 kudos
Just want to confirm - my project uses PDM not poetryand as such uses[project.entry-points.packages]Rather than[tool.poetry.scripts]and the bundle is failing to run on the cluster - as it can't find the entry point - is this expected behavior?
- 9 kudos
- 1449 Views
- 0 replies
- 0 kudos
Azure Databricks job and exception handling
Hi,I'm working on Azure Databricks and I created two jobs, one based on a python wheel and the other based on a notebook, with the same code. The code get data from Azure blob storage, process data with pyspark and send data to EventHub. The whole co...
- 1449 Views
- 0 replies
- 0 kudos
- 3130 Views
- 2 replies
- 2 kudos
Resolved! How do I pass kwargs to wheel method?
Hi,I have a method named main it takes **kwargs as a parameter. def main(**kwargs): parameterOne = kwargs["param-one"] parameterTwo = kwargs["param-two"] parameterThree = kwargs["param-optional-one"] if "param-optional-one" in kwargs else...
- 3130 Views
- 2 replies
- 2 kudos
- 2 kudos
it is command-line parameters so it is like ---param-one=testyou can test it with ArgumentParserfrom argparse import ArgumentParser parser = ArgumentParser() parser.add_argument("--param-one", dest="parameterOne") args = parser.parse_args()
- 2 kudos
- 4950 Views
- 0 replies
- 0 kudos
Adding a dependent library to a Job task permanently adds it to the entire cluster?
Why does adding a dependent library to a Job task also permanently add it to the entire cluster?I am using python wheels, and even when I remove the dependent library from a Job task, the wheel is still part of the cluster configuration.If I then upd...
- 4950 Views
- 0 replies
- 0 kudos