cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Abel_Martinez
by Contributor
  • 6175 Views
  • 9 replies
  • 6 kudos

Resolved! Why I'm getting connection timeout when connecting to MongoDB using MongoDB Connector for Spark 10.x from Databricks

I'm able to connect to MongoDB using org.mongodb.spark:mongo-spark-connector_2.12:3.0.2 and this code:df = spark.read.format("com.mongodb.spark.sql.DefaultSource").option("uri", jdbcUrl)It works well, but if I install last MongoDB Spark Connector ve...

  • 6175 Views
  • 9 replies
  • 6 kudos
Latest Reply
Kaniz
Community Manager
  • 6 kudos

Hi @Abel_Martinez, I want to express my gratitude for your effort in selecting the most suitable solution. It's great to hear that your query has been successfully resolved. Thank you for your contribution. 

  • 6 kudos
8 More Replies
MichelSmits
by New Contributor III
  • 1826 Views
  • 4 replies
  • 3 kudos

Connection to on-prem database occasionally goes to the wrong server

We run a dbx job that uses a jdbc connection to three (SQL Server) on-prem databases. Very occasionally a connection will go to the wrong server.In the job log we see the error "com.microsoft.sqlserver.jdbc.SQLServerException: Login failed for user '...

  • 1826 Views
  • 4 replies
  • 3 kudos
Latest Reply
Anonymous
Not applicable
  • 3 kudos

Hi @Michel Smits​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers ...

  • 3 kudos
3 More Replies
Mani1800
by New Contributor
  • 1012 Views
  • 2 replies
  • 0 kudos

I need to run Sql Update/Delete commands for a AWS RDS system.

I tried 'jdbc' connection to access the data from the RDS. I was able to read the data successfully but I need to do run some update queries. It seems the jdbc won't support update operation. I tried to make connection to my RDS mysql with host, user...

  • 1012 Views
  • 2 replies
  • 0 kudos
Latest Reply
Anonymous
Not applicable
  • 0 kudos

Hi @Manikandan Ramachandran​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear fro...

  • 0 kudos
1 More Replies
Geetha
by New Contributor
  • 1491 Views
  • 2 replies
  • 0 kudos

Connection to Databricks through KNIME

I want to make a connection to Databricks with KNIME. To do this I am using "Create Databricks Environment" node. I have made the following configuration:1. Installed Databricks Simba JDBC driver 2. Made the necessary configuration in Create Databric...

  • 1491 Views
  • 2 replies
  • 0 kudos
Latest Reply
Anonymous
Not applicable
  • 0 kudos

Hi @Geethanjali Nataraj​ Hope everything is going great.Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell...

  • 0 kudos
1 More Replies
Joao_DE
by New Contributor III
  • 1154 Views
  • 3 replies
  • 1 kudos

Information about stoping JDBC connection

Hi everyone!I would like to know how spark stops the connection when reading from a sql database using the JDBC format.Also, if there is a way to check when the connection is active or manually stop it, I also would like to know.Thank you in advance!

  • 1154 Views
  • 3 replies
  • 1 kudos
Latest Reply
Anonymous
Not applicable
  • 1 kudos

Hi @João Peixoto​ Hope everything is going great.Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us so ...

  • 1 kudos
2 More Replies
DeveloperAmarde
by New Contributor
  • 676 Views
  • 1 replies
  • 0 kudos

Connection to Collibra

Hi Team,I want to connect to collibra to fetch details from Collibra.Currently we are using username and password to connect.I want to know recommended practice to connect Collibra account from databricks notebook.

  • 676 Views
  • 1 replies
  • 0 kudos
Latest Reply
Debayan
Esteemed Contributor III
  • 0 kudos

Hi, Could you please know if this helps. https://marketplace.collibra.com/listings/jdbc-driver-for-databricks/

  • 0 kudos
leandro
by New Contributor
  • 534 Views
  • 0 replies
  • 0 kudos

Connection from on-premise R sparklyr session to Databricks, " invalid method toDF for object 17/org.apache.spark.sql.DataFrameReader fields 0 selected 0"

Hello,I'm working with an on-premise R session and would like to connect to Databricks using sparklyr. RStudio server in this case is not an option.I would like to use jdbc. I tested RJDBC + DBI and can conect locally and perform operations. However,...

  • 534 Views
  • 0 replies
  • 0 kudos
mrmota
by New Contributor
  • 817 Views
  • 1 replies
  • 3 kudos

ssh connection with paramiko

I'm trying to use the notebook with python to access our server through ssh. But I am not able to connect through the paramiko library. I have already authorized our server's firewall to accept the databrick ip. Have you ever had a similar case? can ...

  • 817 Views
  • 1 replies
  • 3 kudos
Latest Reply
Aviral-Bhardwaj
Esteemed Contributor III
  • 3 kudos

yes this can happen you have to whitelist that Port in the databricks configuration, there is one spark configuration that you can add to your cluster, that will whitelist your particular portas that configuration can only give by databricks, you can...

  • 3 kudos
elgeo
by Valued Contributor II
  • 6430 Views
  • 7 replies
  • 4 kudos

Resolved! Invalid JDBC url

Hello. I am trying to establish a connection between DBeaver and Databricks. I followed the steps in DBeaver integration with Databricks | Databricks on AWS, but I get the following error while testing the connection: Could anyone provide any insight...

jdbc_url_error
  • 6430 Views
  • 7 replies
  • 4 kudos
Latest Reply
Anonymous
Not applicable
  • 4 kudos

Hi @ELENI GEORGOUSI​ Glad to hear! It's a request that mark an answer as best.Thanks...

  • 4 kudos
6 More Replies
elgeo
by Valued Contributor II
  • 866 Views
  • 1 replies
  • 2 kudos

Databricks - DBeaver error

Hello experts. While trying to setup the connection between DBeaver and Databricks, we receive the following error:[Databricks][DatabricksJDBCDriver](700120) Host adb-xxxxxxxxxx.azuredatabricks.net cannot be resolved through DnsResolver com.databrick...

  • 866 Views
  • 1 replies
  • 2 kudos
Latest Reply
elgeo
Valued Contributor II
  • 2 kudos

We identified the problem. It was due to the proxy that the hostname couldn't be resolved.

  • 2 kudos
Herkimer
by New Contributor II
  • 537 Views
  • 0 replies
  • 0 kudos

intermittent connection error

I am running dbsqlcli in windows 10. I have put together the attached cmd file to pull the identity column data from a series of our tables into individual CSVs so I can upload then to a PostgreSQL DB to do a comparison of each table to those in the ...

  • 537 Views
  • 0 replies
  • 0 kudos
nadia
by New Contributor II
  • 786 Views
  • 1 replies
  • 0 kudos

Resolved! Connection Databricks Postgresql

I use Databricks and I try to connect to posgresql via the following code"jdbcHostname = "xxxxxxx"jdbcDatabase = "xxxxxxxxxxxx"jdbcPort = "5432"username = "xxxxxxx"password = "xxxxxxxx"jdbcUrl = "jdbc:postgresql://{0}:{1}/{2}".format(jdbcHostname, jd...

  • 786 Views
  • 1 replies
  • 0 kudos
Latest Reply
Prabakar
Esteemed Contributor III
  • 0 kudos

hi @Boumaza nadia​ Please check the Ganglia metrics for the cluster. This could be a scalability issue where cluster is overloading. This can happen due to a large partition not fitting into the given executor's memory. To fix this we recommend bump...

  • 0 kudos
Orianh
by Valued Contributor II
  • 1349 Views
  • 2 replies
  • 2 kudos

Resolved! pyodbc read only connection.

Hey Guys, Is there a way to open pyodbc read only connection with simba spark driver? At the moment, I'm able to execute queries such as select , delete, insert into - basically every sql statement using pyodbc. I tried to open pyodbc connection but ...

  • 1349 Views
  • 2 replies
  • 2 kudos
Latest Reply
Hubert-Dudek
Esteemed Contributor III
  • 2 kudos

This readonly=True is working only on some drivers. Just create additional users with granted read-only permission.

  • 2 kudos
1 More Replies
RK_AV
by New Contributor III
  • 668 Views
  • 2 replies
  • 1 kudos

Resolved! Databricks to Databricks connection

How do you connect to Azure Databricks instance from another Databricks instance? I needed to access (database) Views created in a Databricks instance from a Pyspark notebook running in another Databricks instance. Appreciate if anyone has any sample...

  • 668 Views
  • 2 replies
  • 1 kudos
Latest Reply
Kaniz
Community Manager
  • 1 kudos

Hi @Venkata Ramakrishna Alvakonda​ , The two ways of executing a notebook within another notebook in Databricks are:-Method #[1: %run command]​ The first and the most straightforward way of executing another notebook is by using the %run command. Ex...

  • 1 kudos
1 More Replies
Rnmj
by New Contributor III
  • 8534 Views
  • 5 replies
  • 7 kudos

ConnectException: Connection refused (Connection refused) This is often caused by an OOM error

I am trying to run a python code where a json file is flattened to pipe separated file . The code works with smaller files but for huge files of 2.4 GB I get below error:ConnectException: Connection refused (Connection refused)Error while obtaining a...

  • 8534 Views
  • 5 replies
  • 7 kudos
Latest Reply
Rnmj
New Contributor III
  • 7 kudos

Hi @Jose Gonzalez​ , @Werner Stinckens​  @Kaniz Fatma​ ,Thanks for your response .Appreciate a lot. The issue was in the code, it was a python /panda code running on Spark. Due to this only driver node was being used. i did validate this by increasin...

  • 7 kudos
4 More Replies
Labels