cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Help
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Start a conversation
Vindhya
by • New Contributor II
  • 1386 Views
  • 2 replies
  • 0 kudos

Dataframes to Pandas conversion step is failing with exception ""java.lang.IndexOutOfBoundsException: index: 16384, length: 4 (expected: range(0, 16384))"

Dataframes to Pandas conversion step is failing with exception ""java.lang.IndexOutOfBoundsException: index: 16384, length: 4 (expected: range(0, 16384))", PFB screenshot for more details

sccreenshot
Data Engineering
Reply
  • 1386 Views
  • 2 replies
  • 0 kudos
View Replies
Latest Reply
Anonymous
Not applicable
  • 0 kudos

Hi @Vindhya D​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers you...

  • 0 kudos
1 More Replies
kilaki
by • New Contributor II
  • 2354 Views
  • 3 replies
  • 0 kudos

Query fails with 'Error occurred while deserializing arrow data' on Databricks SQL with Channel set to Preview

Noticed with a query based on inline select and joins fails to the client with 'Error occurred while deserializing arrow data'  I.e the query succeeds on Databricks but client (DBeaver, AtScale) receives an errorThe error is only noticed with Databri...

Screen Shot 2023-01-24 at 2.08.54 PM Screen Shot 2023-01-24 at 2.11.20 PM Screen Shot 2023-01-24 at 2.03.21 PM
Data Engineering
Reply
  • 2354 Views
  • 3 replies
  • 0 kudos
View Replies
Latest Reply
franco_patano
New Contributor III
  • 0 kudos

Opened an ES on this, looks like an issue with the Preview channel. Thanks for your help!

  • 0 kudos
2 More Replies
mattmunz
by • New Contributor III
  • 2323 Views
  • 1 replies
  • 4 kudos

JDBC Error: Error occured while deserializing arrow data

I am getting the following error in my Java application.java.sql.SQLException: [Databricks][DatabricksJDBCDriver](500618) Error occured while deserializing arrow data: sun.misc.Unsafe or java.nio.DirectByteBuffer.<init>(long, int) not availableI beli...

Data Engineering
Reply
  • 2323 Views
  • 1 replies
  • 4 kudos
View Replies
Latest Reply
User16753725469
Contributor II
  • 4 kudos

Please try adding the below(--add-opens flag) java command line flags in your jvm call:% javac -classpath SparkJDBC42Example.jar:. jdbc_example.java                   % java --add-opens=java.base/java.nio=ALL-UNNAMED -classpath SparkJDBC42Example.jar...

  • 4 kudos
Labels