cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

dng
by New Contributor III
  • 3463 Views
  • 8 replies
  • 11 kudos

Databricks JDBC Driver v2.6.29 Cloud Fetch failing for Windows Operating System

Hi everyone, I've been stuck for the past two days on this issue with my Databricks JDBC driver and I'm hoping someone can give me more insight into how to troubleshoot. I am using the Databricks JDBC driver in RStudio and the connection was working ...

  • 3463 Views
  • 8 replies
  • 11 kudos
Latest Reply
Prabakar
Esteemed Contributor III
  • 11 kudos

@Debbie Ng​ From your message I see there was a windows update and this failure started. based on the conversation you tried latest version of the driver and still you face the problem. I believe this is something related to the Java version compatib...

  • 11 kudos
7 More Replies
Rajesh_P
by New Contributor II
  • 609 Views
  • 1 replies
  • 1 kudos

Does JDBC and ODBC connection to Databricks Cluster or SQL Warehouse encrypted?

Hello, our consumers (Dell Boomi and other apps) needs data from Databricks. Databricks provides JDBC and ODBC drivers. Does JDBC and ODBC connection to Databricks Cluster or SQL Warehouse encrypted? I am talking about the data in-transit between Dat...

  • 609 Views
  • 1 replies
  • 1 kudos
Latest Reply
Anonymous
Not applicable
  • 1 kudos

Hi @Rajesh Paul​ Great to meet you, and thanks for your question! Let's see if your peers in the community have an answer to your question. Thanks.

  • 1 kudos
Tico23
by Contributor
  • 8390 Views
  • 12 replies
  • 10 kudos

Connecting SQL Server (on-premise) to Databricks via jdbc:sqlserver

Is it possible to connect to SQL Server on-premise (Not Azure) from Databricks?I tried to ping my virtualbox VM (with Windows Server 2022) from within Databricks and the request timed out.%sh   ping 122.138.0.14This is what my connection might look l...

  • 8390 Views
  • 12 replies
  • 10 kudos
Latest Reply
DBXC
Contributor
  • 10 kudos

You need to setup the VNet and wire up the connection between Databricks and on-prem via VPN or ExpressRoute

  • 10 kudos
11 More Replies
PraveenC
by New Contributor II
  • 1596 Views
  • 4 replies
  • 3 kudos

[Databricks][JDBC](10400) Invalid type for data - column: 10, type: Array

Getting below error while mapping an Array Column to String[] entity. Please suggest if Databricks JDBC support entity mapping of Array Values [Worked the same code for below config - H2 DB version - 2.1.214 and org.hibernate.dialect.H2Dialect - ...

  • 1596 Views
  • 4 replies
  • 3 kudos
Latest Reply
Atanu
Esteemed Contributor
  • 3 kudos

Hello @Emmanuel Trindade​  @Praveen C​  This does not look like coming from Databricks end. Look at the error thread.javax.persistence.PersistenceException: org.hibernate.exception.DataException: Could not read entity state from ResultSet : EntityKey...

  • 3 kudos
3 More Replies
Alexander1
by New Contributor III
  • 5039 Views
  • 5 replies
  • 1 kudos

Resolved! Databricks JDBC/ODBC write batch size

I have spent way too much time to find a solution to the problem of efficiently writing data to Databricks via JDBC/ODBC. I have looked into countless docs, blogs and repos and I cannot find one example where someone is setting some kind of batch/bul...

  • 5039 Views
  • 5 replies
  • 1 kudos
Latest Reply
Alexander1
New Contributor III
  • 1 kudos

@Vidula Khanna​ yes, have done so. thanks.

  • 1 kudos
4 More Replies
Sid1805
by New Contributor II
  • 1865 Views
  • 5 replies
  • 0 kudos

Resolved! Calling Delta Tables using JDBC

Hi team,If we kill - clusters every-time will the connection details changes.if yes, If there a way we can mask this so that the End users are not impacted dur to any changes in Clusters.Also if I want to call a Delta Table from an API using JDBC - s...

  • 1865 Views
  • 5 replies
  • 0 kudos
Latest Reply
Anonymous
Not applicable
  • 0 kudos

Hi @Siddharth Krishna​ Hope everything is going great.Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell u...

  • 0 kudos
4 More Replies
haggholm
by New Contributor
  • 1278 Views
  • 2 replies
  • 1 kudos

Resolved! Query with ORDER BY fails with HiveThriftServerError "requirement failed: Subquery … has not finished"

Using ODBC or JDBC to read from a table fails when I attempt to use an ORDER BY clause. In one sample case, I have a fairly small table (just 1946 rows).select * from some_table order by some_fieldResult:java.lang.IllegalArgumentException: requiremen...

  • 1278 Views
  • 2 replies
  • 1 kudos
Latest Reply
Anonymous
Not applicable
  • 1 kudos

Hi @petter@hightouch.com Petter​ Thank you for your question! To assist you better, please take a moment to review the answer and let me know if it best fits your needs.Please help us select the best solution by clicking on "Select As Best" if it doe...

  • 1 kudos
1 More Replies
Optum
by New Contributor III
  • 5158 Views
  • 10 replies
  • 4 kudos

Resolved! Databricks JDBC & Remote Write

Hello,I'm trying to write to a Delta Table in my Databricks instance from a remote Spark session on a different cluster with the Simba Spark driver. I can do reads, but when I attempt to do a write, I get the following error:{  df.write.format("jdbc...

  • 5158 Views
  • 10 replies
  • 4 kudos
Latest Reply
Atanu
Esteemed Contributor
  • 4 kudos

Could you try setting the flag to ignore transactions? I’m not sure what the exact flag is, but there should be more details in the JDBC manual on how to do this

  • 4 kudos
9 More Replies
Akshith_Rajesh
by New Contributor III
  • 2946 Views
  • 4 replies
  • 6 kudos

Unable to write Data frame to Azure Synapse Table

When I am trying to insert records into the azure synapse Table using JDBC Its throwing below error com.microsoft.sqlserver.jdbc.SQLServerException: The statement failed. Column 'COMPANY_ADDRESS_STATE' has a data type that cannot participate ...

  • 2946 Views
  • 4 replies
  • 6 kudos
Latest Reply
Kaniz
Community Manager
  • 6 kudos

Hi @Rajesh Akshith​, We haven't heard from you on the last response from @Hubert Dudek​ , and I was checking back to see if his suggestions helped you. Or else, If you have any solution, please share it with the community as it can be helpful to othe...

  • 6 kudos
3 More Replies
Cano
by New Contributor III
  • 698 Views
  • 1 replies
  • 2 kudos

How to add notebook to my Databricks jdbc url?

Please how do I add a notebook to the jdbc url in order to run queries externally?jdbc:databricks://dbc-a1b2345c-d6e7.cloud.databricks.com:443/default;transportMode=http;ssl=1;httpPath=sql/protocolv1/o/1234567890123456/1234-567890-reef123;AuthMech=3;...

  • 698 Views
  • 1 replies
  • 2 kudos
Latest Reply
ranged_coop
Valued Contributor II
  • 2 kudos

Not sure if it is possible.Alternatively you could try adding your notebook to a job, and then triggering that job via jobs api.Please refer below link Jobs API 2.1 | Databricks on AWS

  • 2 kudos
sriramkumar
by New Contributor II
  • 654 Views
  • 2 replies
  • 1 kudos

Reasons for new Databricks driver

What are the reasons behind Databricks going for their own driver? What differences are made when switching between the previous Spark driver and the new Databricks driver?Is there any specific document I can look at or just the release notes?Also, w...

  • 654 Views
  • 2 replies
  • 1 kudos
Latest Reply
Anonymous
Not applicable
  • 1 kudos

Hey @Sriramkumar Thamizharasan​ Hope all is well! Just wanted to check in if you were able to resolve your issue would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from...

  • 1 kudos
1 More Replies
sriramkumar
by New Contributor II
  • 1780 Views
  • 3 replies
  • 0 kudos

New Databricks Driver gives SQLNonTransientConnectionException when trying to connect to Databricks Instance

import com.databricks.client.jdbc.DataSource;   import java.sql.*;   public class testDatabricks { public static void main(String[] args) throws SQLException { String dbUrl = "jdbc:databricks://<hostname>:443;HttpPath=<HttpPath>;"; // Cop...

  • 1780 Views
  • 3 replies
  • 0 kudos
Latest Reply
Atanu
Esteemed Contributor
  • 0 kudos

This looks like due to maintenance on US . Are you still facing the issue @Sriramkumar Thamizharasan​ Is your workspace on eastus and eastus2 ?

  • 0 kudos
2 More Replies
valdereo
by New Contributor
  • 3904 Views
  • 4 replies
  • 1 kudos

Resolved! net.ucanaccess.jdbc.UcanaccessSQLException: UCAExc:::5.0.1 user lacks privilege or object not found: full questionnaire in statement [SELECT * FROM &#34;full questionnaire&#34; WHERE 1=0]

I have followed the steps given here to parse .accdb files using ucanaccess on Azure Databricks, however, I receive errors.See below my code:# Connection properties conn_properties = {"driver" : "net.ucanaccess.jdbc.UcanaccessDriver"}   # Path url = ...

net.ucanaccess.jdbc.UcanaccessSQLException_ UCAExc___5.0.1 user lacks privilege or object not found_ full questionnaire in statement [SELECT _ FROM  &amp;#34;full questionnaire&amp;#34; WHERE 1=0]
  • 3904 Views
  • 4 replies
  • 1 kudos
Latest Reply
Atanu
Esteemed Contributor
  • 1 kudos

@Valeri Nteleah​ please let me know if you need further help on this.Thanks

  • 1 kudos
3 More Replies
tz1
by New Contributor III
  • 10999 Views
  • 13 replies
  • 7 kudos

Resolved! Problem with Databricks JDBC connection: Error occured while deserializing arrow data

I have a Java program like this to test out the Databricks JDBC connection with the Databricks JDBC driver. Connection connection = null; try { Class.forName(driver); connection = DriverManager.getConnection(url...

  • 10999 Views
  • 13 replies
  • 7 kudos
Latest Reply
Alice__Caterpil
New Contributor III
  • 7 kudos

Hi @Jose Gonzalez​ ,This similar issue in snowflake in JDBC is a good reference, I was able to get this to work in Java OpenJDK 17 by having this JVM option specified:--add-opens=java.base/java.nio=ALL-UNNAMEDAlthough I came across another issue with...

  • 7 kudos
12 More Replies
ninjadev999
by New Contributor II
  • 4276 Views
  • 7 replies
  • 1 kudos

Resolved! Can't write big DataFrame into MSSQL server by using jdbc driver on Azure Databricks

I'm reading a huge csv file including 39,795,158 records and writing into MSSQL server, on Azure Databricks. The Databricks(notebook) is running on a cluster node with 56 GB Memory, 16 Cores, and 12 workers.This is my code in Python and PySpark:from ...

  • 4276 Views
  • 7 replies
  • 1 kudos
Latest Reply
User16764241763
Honored Contributor
  • 1 kudos

Hi,If you are using Azure SQL DB Managed instance, could you please file a support request with Azure team? This is to review any timeouts, perf issues on the backend.Also, it seems like the timeout is coming from SQL Server which is closing the conn...

  • 1 kudos
6 More Replies
Labels