cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

mattmunz
by New Contributor III
  • 17174 Views
  • 6 replies
  • 0 kudos

How can I resolve this SSL error which occurrs when calling databricks-sql-connector/databricks.sql.connect() from my python app?

Error: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:997)>  python --versionPython 3.10.4This error seems to be coming from the thrift backend. I suspect but have not confirmed that t...

  • 17174 Views
  • 6 replies
  • 0 kudos
Latest Reply
hoffman
New Contributor II
  • 0 kudos

One way to resolve this could be to ensure your connection values in are surrounded by quotes. host: "hostname.databricks.com" # Requiredhttp_path: "/sql/1.0/warehouses/aaaabbbccc"   

  • 0 kudos
5 More Replies
Hardy
by New Contributor III
  • 4333 Views
  • 5 replies
  • 4 kudos

The driver could not establish a secure connection to SQL Server by using Secure Sockets Layer (SSL) encryption

I am trying to connect to SQL through JDBC from databricks notebook. (Below is my notebook command)val df = spark.read.jdbc(jdbcUrl, "[MyTableName]", connectionProperties) println(df.schema)When I execute this command, with DBR 10.4 LTS it works fin...

  • 4333 Views
  • 5 replies
  • 4 kudos
Latest Reply
DBXC
Contributor
  • 4 kudos

Try to add the following parameters to your SQL connection string. It fixed my problem for 13.X and 12.X;trustServerCertificate=true;hostNameInCertificate=*.database.windows.net; 

  • 4 kudos
4 More Replies
andr3s
by New Contributor II
  • 7054 Views
  • 3 replies
  • 2 kudos

SSL_connect: certificate verify failed with Power BI

Hi, I'm getting this error with Power BI:Any ideas?Thanks in advance,Andres

Screenshot 2023-05-19 154328
  • 7054 Views
  • 3 replies
  • 2 kudos
Latest Reply
andr3s
New Contributor II
  • 2 kudos

Hi Harrison, thanks for the reply. We have sorted the issue a few days later but forgot to reply to your answer. There had been some changes on the security set-up that affected our connectivity to Databricks from all clients.

  • 2 kudos
2 More Replies
Jayanth746
by New Contributor III
  • 11897 Views
  • 10 replies
  • 4 kudos

Kafka unable to read client.keystore.jks.

Below is the error we have received when trying to read the stream Caused by: kafkashaded.org.apache.kafka.common.KafkaException: Failed to load SSL keystore /dbfs/FileStore/Certs/client.keystore.jksCaused by: java.nio.file.NoSuchFileException: /dbfs...

  • 11897 Views
  • 10 replies
  • 4 kudos
Latest Reply
mwoods
New Contributor III
  • 4 kudos

Ok, scrub that - the problem in my case was that I was using the 14.0 databricks runtime, which appears to have a bug relating to abfss paths here. Switching back to the 13.3 LTS release resolved it for me. So if you're in the same boat finding abfss...

  • 4 kudos
9 More Replies
Rexton
by New Contributor
  • 4020 Views
  • 5 replies
  • 3 kudos

AWS Databricks Pyspark - Unable to connect to Azure MySQL - Shows "SSL Connection is required"

Even after specifying SSL options, unable to connect to MySQL. What could have gone wrong? Could anyone experience similar issues? df_target_master = spark.read.format("jdbc")\.option("driver", "com.mysql.jdbc.Driver")\.option("url", host_url)\.optio...

  • 4020 Views
  • 5 replies
  • 3 kudos
Latest Reply
a2barbosa
New Contributor II
  • 3 kudos

Hey,Here the solution: The correct option for ssl is "useSSL" and not just "ssl".This code below could works:df_target_master = spark.read.format("jdbc")\.option("driver", "com.mysql.jdbc.Driver")\.option("url", host_url)\.option("dbtable", supply_ma...

  • 3 kudos
4 More Replies
SteveGPT
by New Contributor III
  • 2943 Views
  • 6 replies
  • 3 kudos

How to by pass SSL cert verification, using Repos with Azure Devops

Hi all, after some time working with Devops and Repos and getting used to the convenience our SSL Cert situation got jacked up somehow. While not ideal, I'd like to be able to temporarily bypass cert verification. There are ways to do this in the she...

  • 2943 Views
  • 6 replies
  • 3 kudos
Latest Reply
SteveGPT
New Contributor III
  • 3 kudos

Guess I'm out of luck on this one...

  • 3 kudos
5 More Replies
mj2022
by New Contributor III
  • 1963 Views
  • 2 replies
  • 2 kudos

Spark Streaming with SASL_SSL Kafka throwing java.nio.file.NoSuchFileException: dbfs:/mnt/**/kafka.client.truststore.imported.jks

I testing Spark Streaming working withSASL_SSL enabled kafka broker in a notebook.as per this guide https://docs.databricks.com/spark/latest/structured-streaming/kafka.htmli have copied jsk files in an s3 bucket and mounted it in dbfs.In notebook wh...

  • 1963 Views
  • 2 replies
  • 2 kudos
Latest Reply
mj2022
New Contributor III
  • 2 kudos

Thanks..Yes '/dbfs/mnt/xxxx/kafka.client.truststore.imported.jks'  path worked. Also other workaround we got it working, is copy the file from s3 to filesystem using init script and use filepath.

  • 2 kudos
1 More Replies
Rahul_Samant
by Contributor
  • 9598 Views
  • 9 replies
  • 2 kudos

SSL Error While Setting up databricks cli or installing library

How to fix below SSL error while setting up databricks Cli or installing library in clusterLibrary installation attempted on the driver node of cluster *** and failed.Please refer to the following error message to fix the library or contact Databric...

  • 9598 Views
  • 9 replies
  • 2 kudos
Latest Reply
Megan05
New Contributor III
  • 2 kudos

I was getting an SSL when trying to set up secrets using the Databricks CLI. To fix the CLI SSL error I went to %USERPROFILE%\.databrickscfg (~/.databrickscfg on Unix, Linux, or macOS) from the file explorer on my local machine and added the insecure...

  • 2 kudos
8 More Replies
geertvanhove
by New Contributor III
  • 6566 Views
  • 7 replies
  • 6 kudos

Resolved! connection to databricks data source in Visual Studio fails with SSL error

Hi,I'm new to databricks but am positively surprised by the product. We use databricks delta tables as source to build a tabular model, which will serve as data source for Power Bi. To develop our tabular model we use Visual studio to import tables ...

  • 6566 Views
  • 7 replies
  • 6 kudos
Latest Reply
Kaniz_Fatma
Community Manager
  • 6 kudos

Hi @geert vanhove​ ​ , Just a friendly follow-up. Do you still need help, or have you resolved your problem with the above solutions? Please let us know.

  • 6 kudos
6 More Replies
Michael_Galli
by Contributor II
  • 8366 Views
  • 6 replies
  • 3 kudos

Resolved! com.microsoft.sqlserver.jdbc.SQLServerException:The driver could not establish a secure connection to SQL Server by using SSL encr. Error: "Unexpected rethrowing"

Hi all,there is a random error when pushing data from Databricks to a Azure SQL Database.Anyone else also had this problem? Any ideas are appreciated.See stacktrace attached.Target: Azure SQL Database, Standard S6: 400 DTUsDatabricks Cluster config:"...

  • 8366 Views
  • 6 replies
  • 3 kudos
Latest Reply
Michael_Galli
Contributor II
  • 3 kudos

@Pearl Ubaru​ TLS 1.1 is already deprecated.Are there any concerns from your side to set TLS 1.2 in the connection string?

  • 3 kudos
5 More Replies
Prabakar
by Esteemed Contributor III
  • 5529 Views
  • 2 replies
  • 5 kudos

Resolved! %pip/%conda doesn't work with encrypted clusters starting DBR 9.x

While trying to use the magic command %pip/%conda with DBR 9.x or above it fails with the following error:%pip install numpyorg.apache.spark.SparkException: %pip/%conda commands use unencrypted NFS and are disabled by default when SSL encryption is ...

  • 5529 Views
  • 2 replies
  • 5 kudos
Latest Reply
Prabakar
Esteemed Contributor III
  • 5 kudos

If you are not aware of the traffic encryption between cluster worker nodes, you can refer to the below link.https://docs.microsoft.com/en-us/azure/databricks/security/encryption/encrypt-otw

  • 5 kudos
1 More Replies
RantoB
by Valued Contributor
  • 5779 Views
  • 5 replies
  • 0 kudos

Resolved! SSLCertVerificationError how to disable SSL Certification

Hi, How is that possible to disable SSL Certification.With databricks API I got this error :SSLCertVerificationError   SSLCertVerificationError: ("hostname 'https' doesn't match either of '*.numericable.fr', 'numericable.fr'",)   MaxRetryError: HTTPS...

  • 5779 Views
  • 5 replies
  • 0 kudos
Latest Reply
Anonymous
Not applicable
  • 0 kudos

@Bertrand BURCKER​ - Thanks for letting us know your issue is resolved. If @Prabakar Ammeappin​'s answer solved the problem, would you be happy to mark his answer as best so others can more easily find an answer for this?

  • 0 kudos
4 More Replies
akj2784
by New Contributor II
  • 17111 Views
  • 11 replies
  • 1 kudos

How to connect PostgreSQL from Databricks

I am trying to connect PostgreSQL from Azure Databricks. I am using the below code to connect. jdbcHostname = "Test" jdbcPort = 1234 jdbcDatabase = "Test1" jdbcUrl = "jdbc:postgresql://{0}:{1}/{2}".format(jdbcHostname, jdbcPort, jdbcDatabase) Conn...

  • 17111 Views
  • 11 replies
  • 1 kudos
Latest Reply
Anonymous
Not applicable
  • 1 kudos

@Javier De La Torre do you really need two-way SSL (verify-full)? In most cases one way SSL (sslmode=require) should be enough. @akj2784​  When you say "Connection was successful", where do you mean you established a successful connection? You might...

  • 1 kudos
10 More Replies
cfregly
by Contributor
  • 4782 Views
  • 4 replies
  • 0 kudos
  • 4782 Views
  • 4 replies
  • 0 kudos
Latest Reply
TianziCai
New Contributor II
  • 0 kudos

sample = (spark.read .format("com.databricks.spark.redshift") .option("url", jdbcUrl) .option("dbtable", "xx.xxx") # schema, table .option("forward_spark_s3_credentials", True) .option("tempdir", tem...

  • 0 kudos
3 More Replies
Labels