cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

Incorrect length for `string` returned by the Databricks ODBC driver

Hรฅkon_ร…mdal
New Contributor III

Dear Databricks and community,

โ€‹

I have been struggling with a bug related to using golang and the Databricks ODBC driver.

โ€‹

It turns out that `SQLDescribeColW` consequently returns 256 as a length for `string` columns. However, in Spark, strings might be much longer than that.

This is normally not an issue unless the ODBC library actively uses this information when allocating and reading data. This is the case for alexbrainman/odbc, which allocates 256 bytes for the strings and seg-faults when it reads cells where the strings are longer than 256 bytes.

The issue, and the fix, is described in detail here: https://github.com/alexbrainman/odbc/issues/165

As far as I know, the driver is developed by Magnitude (https://www.magnitude.com/drivers/spark-jdbc-odbc) , so I will see if there is a way for me to let them know about this bug. Still, since the driver is distributed by Databricks, I felt a post/bug report would be beneficial in this repository as well.

Thank you!

Hรฅkon

1 ACCEPTED SOLUTION

Accepted Solutions

User16829050420
Databricks Employee
Databricks Employee

Thanks for posting this issue @Hรฅkon ร…mdalโ€‹ . We should be able to reproduce and report it to the Magnitude team subsquently.

View solution in original post

1 REPLY 1

User16829050420
Databricks Employee
Databricks Employee

Thanks for posting this issue @Hรฅkon ร…mdalโ€‹ . We should be able to reproduce and report it to the Magnitude team subsquently.

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโ€™t want to miss the chance to attend and share knowledge.

If there isnโ€™t a group near you, start one and help create a community that brings people together.

Request a New Group