Databricks 2.6.25 JDBC driver can't create tables with `GENERATED` columns

findinpath — Fri, 06 May 2022 08:38:30 GMT

I'm using the Databricks JDBC driver recently made available via Maven:

https://mvnrepository.com/artifact/com.databricks/databricks-jdbc/2.6.25

While trying to create a table with `GENERATED` columns I receive the following exception:

Caused by: java.sql.SQLException: [Databricks][DatabricksJDBCDriver](500051) ERROR processing query/statement. Error Code: 0, SQL state: Error running query: org.apache.spark.sql.catalyst.parser.ParseException: 
no viable alternative at input 'CREATE TABLE default.test_generated_columns_not_supported_sbju2lhx0065(  id INT,  a_number INT,  a_number_times_two INT GENERATED'(line 1, pos 120)
 
== SQL ==
CREATE TABLE default.test_generated_columns_not_supported_sbju2lhx0065(  id INT,  a_number INT,  a_number_times_two INT GENERATED ALWAYS AS (a_number * 2)) USING DELTA
------------------------------------------------------------------------------------------------------------------------^^^
, Query: CREATE TABLE default.test_generated_columns_not_supported_sbju2lhx0065(  id INT,  a_number INT,  a_number_times_two INT GENERATED ALWAYS AS (a_number * 2)) USING DELTA.
	at com.databricks.client.hivecommon.api.HS2Client.buildExceptionFromTStatusSqlState(Unknown Source)
	at com.databricks.client.hivecommon.api.HS2Client.pollForOperationCompletion(Unknown Source)
	at com.databricks.client.hivecommon.api.HS2Client.executeStatementInternal(Unknown Source)
	at com.databricks.client.hivecommon.api.HS2Client.executeStatement(Unknown Source)
	at com.databricks.client.hivecommon.dataengine.HiveJDBCNativeQueryExecutor.executeRowCountQueryHelper(Unknown Source)
	at com.databricks.client.hivecommon.dataengine.HiveJDBCNativeQueryExecutor.execute(Unknown Source)
	at com.databricks.client.jdbc.common.SStatement.executeNoParams(Unknown Source)
	at com.databricks.client.jdbc.common.BaseStatement.execute(Unknown Source)
	at com.databricks.client.hivecommon.jdbc42.Hive42Statement.execute(Unknown Source)

This query was working fine when I was using simba spark-jdbc42 2.6.16.

I've tried running the query also in a Databricks Community Cloud Notebook and there it was executed just fine.

Is this a regression issue?

Re: Databricks 2.6.25 JDBC driver can't create tables with `GENERATED` columns

Hubert-Dudek — Fri, 06 May 2022 15:43:59 GMT

@Kaniz Fatma @Vartika Nain can we report it as a BUG?

Re: Databricks 2.6.25 JDBC driver can't create tables with `GENERATED` columns

findinpath — Thu, 19 May 2022 04:43:40 GMT

I was under the impression that this has been recognised as a BUG and is being handled by Databricks.

What do I need to do for reporting the issue officially as a BUG?

topic Re: Databricks 2.6.25 JDBC driver can't create tables with `GENERATED` columns in Data Engineering

Databricks 2.6.25 JDBC driver can't create tables with `GENERATED` columns

Re: Databricks 2.6.25 JDBC driver can't create tables with `GENERATED` columns

Re: Databricks 2.6.25 JDBC driver can't create tables with `GENERATED` columns