03-23-2022 01:22 PM
We are using PHP and the Databricks SQL ODBC driver and cannot run a query that users DATE functions with prepared statements. Sample script/docker setup here:
https://github.com/rlorenzo/databricks_php/blob/odbc_prepare_error/test_connection.php
For example, the following query works in with the Python SQL driver:
SELECT count(1), companyid, company_name as name, series, investment_date
FROM refactored_db.view_afd
WHERE companyid not in (
SELECT companyid
FROM refactored_db.view_afd
WHERE investment_date < DATE(NOW()-INTERVAL 64 DAY))
AND investment_date BETWEEN DATE(NOW()-INTERVAL 64 DAY) AND DATE(NOW())
GROUP BY companyid, company_name, series, investment_date
HAVING count(1) <= ?
But using PHP and the ODBC driver we get the error:
Warning: odbc_prepare(): SQL error: [Simba][Hardy] (80) Syntax or semantic analysis error thrown in server while executing query. Error message from server: org.apache.hive.service.cli.HiveSQLException: Error running query: org.apache.spark.sql.catalyst.parser.ParseException:
mismatched input '?' expecting {'(', '{', 'APPLY', 'CALLED', 'CHANGES', 'CLONE', 'COLLECT', 'CONTAINS', 'CONVERT', 'COPY', 'COPY_OPTIO, SQL state 37000 in SQLPrepare in /test_connection.php on line 43
odbc_prepare Error: [Simba][Hardy] (80) Syntax or semantic analysis error thrown in server while executing query. Error message from server: org.apache.hive.service.cli.HiveSQLException: Error running query: org.apache.spark.sql.catalyst.parser.ParseException:
mismatched input '?' expecting {'(', '{', 'APPLY', 'CALLED', 'CHANGES', 'CLONE', 'COLLECT', 'CONTAINS', 'CONVERT', 'COPY', 'COPY_OPTIO
Is this a known issue? We would like to use prepared statements to protect against possible SQL injection. However, it seems we need to forgo prepared statements if the driver doesn't support it fully.
04-09-2022 02:53 AM
Darn it. In cases where rewriting a query is too tedious, another option is defining the query as a logical view and simply selecting from it and applying parameters as needed. With this approach the query in the definition can’t contain any parameters; they must be defined outside the view. Did you give that a shot?
03-26-2022 09:52 AM
@Rex Lorenzo I'll dig into this and find out.
03-31-2022 11:57 PM
Hi Community, Just FYI.
A prepared statement is a feature used to execute the same (or similar) SQL statements repeatedly with high efficiency.
Prepared statements basically work like this:
Compared to executing SQL statements directly, prepared statements have three main advantages:
04-01-2022 12:14 AM
@Rex Lorenzo apologies it took me so long to get back to you. This is definitely a bug, and it is in the ODBC driver. We are aware of it and are working on it. The root cause is that the SQL parser on driver isn't fully aware of Spark SQL i.e. it's best to stick to SQL-92 e.g. you are using DATE(), which is not standard SQL. Instead, try using CAST(as date) or CONVERT( as date).
Can you please try it and let me know what happens?
04-01-2022 11:10 AM
@Bilal Aslam We tried CAST and CONVERT and still getting the same error.
04-09-2022 02:53 AM
Darn it. In cases where rewriting a query is too tedious, another option is defining the query as a logical view and simply selecting from it and applying parameters as needed. With this approach the query in the definition can’t contain any parameters; they must be defined outside the view. Did you give that a shot?
07-12-2022 04:25 AM
Hi @Rex Lorenzo , We haven't heard from you on the last response from @Bilal Aslam , and I was checking back to see if his suggestions helped you. Or else, If you have any solution, please share it with the community as it can be helpful to others.
Also, please don't forget to click on the "Select As Best" button whenever the information provided helps resolve your question.
Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections.
Click here to register and join today!
Engage in exciting technical discussions, join a group with your peers and meet our Featured Members.