Databricks Community

Håkon_Åmdal · 10-25-2021

Dear Databricks and community,I have been struggling with a bug related to using golang and the Databricks ODBC driver.It turns out that `SQLDescribeColW` consequently returns 256 as a length for `string` columns. However, in Spark, strings might b...

Håkon_Åmdal · 12-12-2022

Unless you compress the entire CSV, which also should be a viable approach.That said, Delta/Parquet would normally be the better option where each column in compressed.

Håkon_Åmdal · 05-24-2022

Did you try storing it all to a DELTA table with a MERGE INTO [1]? You can optionally specify a condition on "WHEN MATCHED" such that you only insert if the timestamp is newer.[1] https://docs.databricks.com/spark/latest/spark-sql/language-manual/del...

Håkon_Åmdal · 05-24-2022

Hi. As you say, you can use JDBC/ODBC to connect to the SQL endpoints with the drivers supplied by Databricks [1] [2]. For the programming language you want to use, use a JDBC interface/library to connect to the endpoints. For Java, this is built in ...

Databricks Community

User Stats

User Activity

Incorrect length for `string` returned by the Databricks ODBC driver

Re: How to optimize storage for sparse data in data lake?

Re: How to take only the most recent record from a variable number of tables in a stream

Re: Connect to Databricks SQL Endpoint using Programming language