- 14612 Views
- 12 replies
- 10 kudos
Connecting SQL Server (on-premise) to Databricks via jdbc:sqlserver
Is it possible to connect to SQL Server on-premise (Not Azure) from Databricks?I tried to ping my virtualbox VM (with Windows Server 2022) from within Databricks and the request timed out.%sh ping 122.138.0.14This is what my connection might look l...
- 14612 Views
- 12 replies
- 10 kudos
- 10 kudos
I tried to connect to localhost sql server through databricks community edition, but it failed. I have created an IP rule on port 1433 allowed inbound connection from all public network, but still didn't connect. I tried locally using python its work...
- 10 kudos
- 7187 Views
- 6 replies
- 10 kudos
Databricks JDBC Driver v2.6.29 Cloud Fetch failing for Windows Operating System
Hi everyone, I've been stuck for the past two days on this issue with my Databricks JDBC driver and I'm hoping someone can give me more insight into how to troubleshoot. I am using the Databricks JDBC driver in RStudio and the connection was working ...
- 7187 Views
- 6 replies
- 10 kudos
- 10 kudos
@Debbie Ng​ From your message I see there was a windows update and this failure started. based on the conversation you tried latest version of the driver and still you face the problem. I believe this is something related to the Java version compatib...
- 10 kudos
- 1188 Views
- 1 replies
- 1 kudos
Does JDBC and ODBC connection to Databricks Cluster or SQL Warehouse encrypted?
Hello, our consumers (Dell Boomi and other apps) needs data from Databricks. Databricks provides JDBC and ODBC drivers. Does JDBC and ODBC connection to Databricks Cluster or SQL Warehouse encrypted? I am talking about the data in-transit between Dat...
- 1188 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi @Rajesh Paul​ Great to meet you, and thanks for your question! Let's see if your peers in the community have an answer to your question. Thanks.
- 1 kudos
- 3157 Views
- 4 replies
- 3 kudos
[Databricks][JDBC](10400) Invalid type for data - column: 10, type: Array
Getting below error while mapping an Array Column to String[] entity. Please suggest if Databricks JDBC support entity mapping of Array Values [Worked the same code for below config - H2 DB version - 2.1.214 and org.hibernate.dialect.H2Dialect - ...
- 3157 Views
- 4 replies
- 3 kudos
- 3 kudos
Hello @Emmanuel Trindade​ @Praveen C​ This does not look like coming from Databricks end. Look at the error thread.javax.persistence.PersistenceException: org.hibernate.exception.DataException: Could not read entity state from ResultSet : EntityKey...
- 3 kudos
- 10317 Views
- 5 replies
- 1 kudos
Resolved! Databricks JDBC/ODBC write batch size
I have spent way too much time to find a solution to the problem of efficiently writing data to Databricks via JDBC/ODBC. I have looked into countless docs, blogs and repos and I cannot find one example where someone is setting some kind of batch/bul...
- 10317 Views
- 5 replies
- 1 kudos
- 6732 Views
- 5 replies
- 0 kudos
Resolved! Calling Delta Tables using JDBC
Hi team,If we kill - clusters every-time will the connection details changes.if yes, If there a way we can mask this so that the End users are not impacted dur to any changes in Clusters.Also if I want to call a Delta Table from an API using JDBC - s...
- 6732 Views
- 5 replies
- 0 kudos
- 0 kudos
Hi @Siddharth Krishna​ Hope everything is going great.Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell u...
- 0 kudos
- 2601 Views
- 2 replies
- 1 kudos
Resolved! Query with ORDER BY fails with HiveThriftServerError "requirement failed: Subquery … has not finished"
Using ODBC or JDBC to read from a table fails when I attempt to use an ORDER BY clause. In one sample case, I have a fairly small table (just 1946 rows).select * from some_table order by some_fieldResult:java.lang.IllegalArgumentException: requiremen...
- 2601 Views
- 2 replies
- 1 kudos
- 1 kudos
Hi @petter@hightouch.com Petter​ Thank you for your question! To assist you better, please take a moment to review the answer and let me know if it best fits your needs.Please help us select the best solution by clicking on "Select As Best" if it doe...
- 1 kudos
- 9473 Views
- 8 replies
- 4 kudos
Databricks JDBC & Remote Write
Hello,I'm trying to write to a Delta Table in my Databricks instance from a remote Spark session on a different cluster with the Simba Spark driver. I can do reads, but when I attempt to do a write, I get the following error:{ df.write.format("jdbc...
- 9473 Views
- 8 replies
- 4 kudos
- 4 kudos
Could you try setting the flag to ignore transactions? I’m not sure what the exact flag is, but there should be more details in the JDBC manual on how to do this
- 4 kudos
- 4857 Views
- 3 replies
- 6 kudos
Unable to write Data frame to Azure Synapse Table
When I am trying to insert records into the azure synapse Table using JDBC Its throwing below error com.microsoft.sqlserver.jdbc.SQLServerException: The statement failed. Column 'COMPANY_ADDRESS_STATE' has a data type that cannot participate ...
- 4857 Views
- 3 replies
- 6 kudos
- 6 kudos
Columns that use any of the following data types cannot be included in a columnstore index:nvarchar(max), varchar(max), and varbinary(max) (Applies to SQL Server 2016 and prior versions, and nonclustered columnstore indexes)so the issue is on the Azu...
- 6 kudos
- 1392 Views
- 1 replies
- 2 kudos
How to add notebook to my Databricks jdbc url?
Please how do I add a notebook to the jdbc url in order to run queries externally?jdbc:databricks://dbc-a1b2345c-d6e7.cloud.databricks.com:443/default;transportMode=http;ssl=1;httpPath=sql/protocolv1/o/1234567890123456/1234-567890-reef123;AuthMech=3;...
- 1392 Views
- 1 replies
- 2 kudos
- 2 kudos
Not sure if it is possible.Alternatively you could try adding your notebook to a job, and then triggering that job via jobs api.Please refer below link Jobs API 2.1 | Databricks on AWS
- 2 kudos
- 1350 Views
- 2 replies
- 1 kudos
Reasons for new Databricks driver
What are the reasons behind Databricks going for their own driver? What differences are made when switching between the previous Spark driver and the new Databricks driver?Is there any specific document I can look at or just the release notes?Also, w...
- 1350 Views
- 2 replies
- 1 kudos
- 1 kudos
Hey @Sriramkumar Thamizharasan​ Hope all is well! Just wanted to check in if you were able to resolve your issue would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from...
- 1 kudos
- 3169 Views
- 3 replies
- 0 kudos
New Databricks Driver gives SQLNonTransientConnectionException when trying to connect to Databricks Instance
import com.databricks.client.jdbc.DataSource; import java.sql.*; public class testDatabricks { public static void main(String[] args) throws SQLException { String dbUrl = "jdbc:databricks://<hostname>:443;HttpPath=<HttpPath>;"; // Cop...
- 3169 Views
- 3 replies
- 0 kudos
- 0 kudos
This looks like due to maintenance on US . Are you still facing the issue @Sriramkumar Thamizharasan​ Is your workspace on eastus and eastus2 ?
- 0 kudos
- 7020 Views
- 2 replies
- 1 kudos
Resolved! net.ucanaccess.jdbc.UcanaccessSQLException: UCAExc:::5.0.1 user lacks privilege or object not found: full questionnaire in statement [SELECT * FROM "full questionnaire" WHERE 1=0]
I have followed the steps given here to parse .accdb files using ucanaccess on Azure Databricks, however, I receive errors.See below my code:# Connection properties conn_properties = {"driver" : "net.ucanaccess.jdbc.UcanaccessDriver"} # Path url = ...
![net.ucanaccess.jdbc.UcanaccessSQLException_ UCAExc___5.0.1 user lacks privilege or object not found_ full questionnaire in statement [SELECT _ FROM &#34;full questionnaire&#34; WHERE 1=0]](/t5/image/serverpage/image-id/1952iDBC96B99046D53B6/image-size/medium?v=v2&px=400)
- 7020 Views
- 2 replies
- 1 kudos
- 1 kudos
@Valeri Nteleah​ please let me know if you need further help on this.Thanks
- 1 kudos
- 19112 Views
- 7 replies
- 6 kudos
Resolved! Problem with Databricks JDBC connection: Error occured while deserializing arrow data
I have a Java program like this to test out the Databricks JDBC connection with the Databricks JDBC driver. Connection connection = null; try { Class.forName(driver); connection = DriverManager.getConnection(url...
- 19112 Views
- 7 replies
- 6 kudos
- 6 kudos
Hi @Jose Gonzalez​ ,This similar issue in snowflake in JDBC is a good reference, I was able to get this to work in Java OpenJDK 17 by having this JVM option specified:--add-opens=java.base/java.nio=ALL-UNNAMEDAlthough I came across another issue with...
- 6 kudos
- 6984 Views
- 7 replies
- 1 kudos
Resolved! Can't write big DataFrame into MSSQL server by using jdbc driver on Azure Databricks
I'm reading a huge csv file including 39,795,158 records and writing into MSSQL server, on Azure Databricks. The Databricks(notebook) is running on a cluster node with 56 GB Memory, 16 Cores, and 12 workers.This is my code in Python and PySpark:from ...
- 6984 Views
- 7 replies
- 1 kudos
- 1 kudos
Hi,If you are using Azure SQL DB Managed instance, could you please file a support request with Azure team? This is to review any timeouts, perf issues on the backend.Also, it seems like the timeout is coming from SQL Server which is closing the conn...
- 1 kudos