- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-21-2024 02:39 PM
Hello all,
I am attempting to use named parameter markers as shown in this article: https://docs.databricks.com/en/sql/language-manual/sql-ref-parameter-marker.html#named-parameter-mar...
I can pass strings and numbers in perfectly fine, but the issues arise when I try to pass in arrays. An example:
spark.sql("SELECT 1 IN (1, 2, 3)", args=params).show()
Works just fine, and the function is able to run. However:
spark.sql("SELECT 1 IN :test", args={"test": [1, 2, 3]}).show()
and also
spark.sql("SELECT 1 IN ARRAY(:test)", args={"test": [1, 2, 3]}).show()
give me vague syntax errors, and even just trying to reference the passed-in array in a basic sense like
spark.sql("SELECT :test", args={"test": [1, 2, 3]}).show()
gives the error "[INVALID_SQL_ARG] The argument test of 'sql()' is invalid. Consider to replace it by a SQL literal."
Is there any way to pass in an array using named parameters to pyspark.sql()? Thanks
- Labels:
-
Spark
Accepted Solutions
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-23-2024 07:56 PM - edited 06-23-2024 09:06 PM
Hello @runninsavvy ,
The following code sample can be used in such case
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-23-2024 07:56 PM - edited 06-23-2024 09:06 PM
Hello @runninsavvy ,
The following code sample can be used in such case
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
06-24-2024 06:51 AM
That worked, thanks!

