cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Using Databricks asset bundles with typer instead of argparse

VicS
Visitor

I want to use Databricks asset bundles - I'd like to use `typer` as a CLI tool, but I have only been able to set it up with `argparse`. Argparse seems to be able to retrieve the arguments from the databricks task, but not typer.


I specified two entrypoints in my pyproject.toml

[tool.poetry.scripts]
mypackage_ep_typer = "my_package.entrypoint_typer:main"
mypackage_ep_argparse = "my_package.entrypoint_argparse:entrypoint_generic"


I'm able to use the asset bundles with an entrypoint to my ELTL applications with argparse as follows in the entrypoint_argparse.py:

def entrypoint_generic():
	"""Execute the application."""
	logger.info("Executing 'argparse' entrypoint...")
	parser = argparse.ArgumentParser(description="My module.")
	parser.add_argument(
		"--applicationname",
		help="The name of the application to execute.",
		type=str,
		required=True,
		dest="applicationname",
	)
	args = parser.parse_args()
	logger.info(f"{args.applicationname=}")

This one deploys and runs without any issues in the asset bundle's workflow task.

However, if I try the same with typer in entrypoint_typer.py, I cannot get it to work:

def main(
	applicationname: Annotated[str, typer.Option(help="Application to execute.")]
	):
	"""Execute the application."""
	logger.info("Executing 'typer' entrypoint...")
	logger.info(f"{applicationname=}")

I can run typer locally:

> poetry run python -m typer .\entrypoint_typer.py run --applicationname MYAPPNAME
2024-11-06 xx:xx:xx - root - INFO - Executing 'typer' entrypoint...
2024-11-06 xx:xx:xx - root - INFO - applicationname='MYAPPNAME'

But when I try to deploy and run my asset bundle, I get this error when the workflow task tries to start:

TypeError: main() missing 1 required positional argument: 'applicationname'

But I can see the parameter in the UI...

DdqYAyd4

0 REPLIES 0

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group