Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
10-06-2023 04:12 AM
Hi @Retired_mod, thank you for the suggestion. I tried calling the parameter instead of the widget value. However, as I expected, this didn’t resolve the original problem of getArgument() not being recognised as a function.
query:
folder_path = dbutils.widgets.get('folderPath')
Spark.sql(“””
OPTIMIZE <table>
where date = replace(regexp_extract(getArgument('folder_path'), '=\\\d{8}', 0), '=', '') “””)
error:
Exception: Undefined function: getArgument. This function is neither a built-in/temporary function, nor a persistent function that is qualified as spark_catalog.default.getargument.; line 2 pos 49
