cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Data Engineering
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

Incorrect length for `string` returned by the Databricks ODBC driver

Hรฅkon_ร…mdal
New Contributor III

Dear Databricks and community,

โ€‹

I have been struggling with a bug related to using golang and the Databricks ODBC driver.

โ€‹

It turns out that `SQLDescribeColW` consequently returns 256 as a length for `string` columns. However, in Spark, strings might be much longer than that.

This is normally not an issue unless the ODBC library actively uses this information when allocating and reading data. This is the case for alexbrainman/odbc, which allocates 256 bytes for the strings and seg-faults when it reads cells where the strings are longer than 256 bytes.

The issue, and the fix, is described in detail here: https://github.com/alexbrainman/odbc/issues/165

As far as I know, the driver is developed by Magnitude (https://www.magnitude.com/drivers/spark-jdbc-odbc) , so I will see if there is a way for me to let them know about this bug. Still, since the driver is distributed by Databricks, I felt a post/bug report would be beneficial in this repository as well.

Thank you!

Hรฅkon

1 ACCEPTED SOLUTION

Accepted Solutions

User16829050420
New Contributor III
New Contributor III

Thanks for posting this issue @Hรฅkon ร…mdalโ€‹ . We should be able to reproduce and report it to the Magnitude team subsquently.

View solution in original post

1 REPLY 1

User16829050420
New Contributor III
New Contributor III

Thanks for posting this issue @Hรฅkon ร…mdalโ€‹ . We should be able to reproduce and report it to the Magnitude team subsquently.

Welcome to Databricks Community: Lets learn, network and celebrate together

Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections. 

Click here to register and join today! 

Engage in exciting technical discussions, join a group with your peers and meet our Featured Members.