Cannot use prepared statements with date functions

[Rex]

We are using PHP and the Databricks SQL ODBC driver and cannot run a query that users DATE functions with prepared statements. Sample script/docker setup here:

https://github.com/rlorenzo/databricks_php/blob/odbc_prepare_error/test_connection.php

For example, the following query works in with the Python SQL driver:

SELECT count(1), companyid, company_name as name, series, investment_date
    FROM refactored_db.view_afd
    WHERE companyid not in (
        SELECT companyid
            FROM refactored_db.view_afd
            WHERE investment_date < DATE(NOW()-INTERVAL 64 DAY))
    AND investment_date BETWEEN DATE(NOW()-INTERVAL 64 DAY) AND DATE(NOW())
    GROUP BY companyid, company_name, series, investment_date
    HAVING count(1) <= ?

But using PHP and the ODBC driver we get the error:

Warning: odbc_prepare(): SQL error: [Simba][Hardy] (80) Syntax or semantic analysis error thrown in server while executing query. Error message from server: org.apache.hive.service.cli.HiveSQLException: Error running query: org.apache.spark.sql.catalyst.parser.ParseException: 
mismatched input '?' expecting {'(', '{', 'APPLY', 'CALLED', 'CHANGES', 'CLONE', 'COLLECT', 'CONTAINS', 'CONVERT', 'COPY', 'COPY_OPTIO, SQL state 37000 in SQLPrepare in /test_connection.php on line 43
odbc_prepare Error: [Simba][Hardy] (80) Syntax or semantic analysis error thrown in server while executing query. Error message from server: org.apache.hive.service.cli.HiveSQLException: Error running query: org.apache.spark.sql.catalyst.parser.ParseException: 
mismatched input '?' expecting {'(', '{', 'APPLY', 'CALLED', 'CHANGES', 'CLONE', 'COLLECT', 'CONTAINS', 'CONVERT', 'COPY', 'COPY_OPTIO

Is this a known issue? We would like to use prepared statements to protect against possible SQL injection. However, it seems we need to forgo prepared statements if the driver doesn't support it fully.

[BilalAslamDbrx]

@Rex Lorenzo​ I'll dig into this and find out.

[BilalAslamDbrx]

@Rex Lorenzo​ apologies it took me so long to get back to you. This is definitely a bug, and it is in the ODBC driver. We are aware of it and are working on it. The root cause is that the SQL parser on driver isn't fully aware of Spark SQL i.e. it's best to stick to SQL-92 e.g. you are using DATE(), which is not standard SQL. Instead, try using CAST(as date) or  CONVERT( as date).

Can you please try it and let me know what happens?

[Rex]

@Bilal Aslam​ We tried CAST and CONVERT and still getting the same error.

[BilalAslamDbrx · Accepted solution]

Darn it. In cases where rewriting a query is too tedious, another option is defining the query as a logical view and simply selecting from it and applying parameters as needed. With this approach the query in the definition can’t contain any parameters; they must be defined outside the view. Did you give that a shot?