cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Help
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Start a conversation
BorislavBlagoev
by • Valued Contributor III
  • 27902 Views
  • 33 replies
  • 14 kudos

Hello, everyone. I want to ask if there is a way to connect Databricks cluster with SSH interpreter in your IDE? I know about databricks connect but I want to execute the entire code in the cluster.

Data Engineering
Reply
  • 27902 Views
  • 33 replies
  • 14 kudos
View Replies
Latest Reply
bhuvahh
New Contributor II
  • 14 kudos

I think plain python code will run with databricks connect (if it is a python program you are writing), and spark sql can be done by spark.sql(...).

  • 14 kudos
32 More Replies
Sascha
by • New Contributor III
  • 5457 Views
  • 4 replies
  • 2 kudos

Resolved! Unable to connect to Confluent from Databricks

I'm facing the same issue as this post: https://community.databricks.com/s/question/0D58Y00009DE82zSAD/databricks-kafka-read-not-connectingIn my case I'm connecting to Confluent Cloud. I'm able to ping the bootstrap server, I'm able to netstat succes...

Data Engineering
Reply
  • 5457 Views
  • 4 replies
  • 2 kudos
View Replies
Latest Reply
Sascha
New Contributor III
  • 2 kudos

Hi @Debayan Mukherjee​ , no I haven't.But with the help of Confluent I changed the statement to the below, and somehow this solved it.inputDF = (spark .readStream .format("kafka") .option("kafka.bootstrap.servers", host) .option("kafka.ssl.en...

  • 2 kudos
3 More Replies
Jayesh
by • New Contributor III
  • 2702 Views
  • 5 replies
  • 3 kudos

Resolved! How can we do data copy from Databricks SQL using notebook?

Hi Team, we have a scenario where we have to connect to the DataBricks SQL instance 1 from another DataBricks instance 2 using notebook or Azure Data Factory. Can you please help?

Data Engineering
Reply
  • 2702 Views
  • 5 replies
  • 3 kudos
View Replies
Latest Reply
Anonymous
Not applicable
  • 3 kudos

Thanks for jumping in to help @Arvind Ravish​  @Hubert Dudek​ and @Artem Sheiko​ !

  • 3 kudos
4 More Replies
AP
by • New Contributor III
  • 2483 Views
  • 2 replies
  • 2 kudos

How can we connect to the databricks managed metastore

Hi, I am trying to take advantage of the treasure trove of the information that metastore contains and take some actions to improve performance. In my case, the metastore is managed by databricks, we don't use external metastore.How can I connect to ...

Data Engineering
Reply
  • 2483 Views
  • 2 replies
  • 2 kudos
View Replies
Latest Reply
Prabakar
Databricks Employee
  • 2 kudos

@AKSHAY PALLERLA​ to get the jdbc/odbc information you can get it from the cluster configuration. In the cluster configuration page, under advanced options, you have JDBC/ODBC tab. Click on that tab and it should give you the details you are looking ...

  • 2 kudos
1 More Replies
avinash_goje
by • New Contributor II
  • 2603 Views
  • 2 replies
  • 2 kudos

How to send metrics from GCP Databricks to Grafana Cloud through Prometheus?

While connecting the Databricks and Grafana, I have gone through the following approach.Install Grafna Agent in Databrics Clusters from Databricks console --> Not working since the system is not booted with systemd as init systemSince Spark 3 has Pro...

Data Engineering
Reply
  • 2603 Views
  • 2 replies
  • 2 kudos
View Replies
Latest Reply
Hubert-Dudek
Esteemed Contributor III
  • 2 kudos

There is a repo with Prometheus gateway https://gist.github.com/Lowess/3a71792d2d09e38bf8f524644bbf8349. In the community, we usually use DataDog as both plays nicely https://docs.datadoghq.com/integrations/databricks/?tabs=driveronly

  • 2 kudos
1 More Replies
Vadim1
by • New Contributor III
  • 2606 Views
  • 3 replies
  • 1 kudos

Resolved! Connect from Databricks to Hbase HDinsight cluster.

Hi, I have Databricks installation in Azure. I want to run a job that connects to HBase in a separate HDinsight cluster.What I tried:Created a peering between base cluster and Databricks vNets.I can ping IPs of Hbase zookeeper nodes but I cannot acce...

Data Engineering
Reply
  • 2606 Views
  • 3 replies
  • 1 kudos
View Replies
Latest Reply
User16764241763
Honored Contributor
  • 1 kudos

Vadim, Thank you for the response. Appreciate it.

  • 1 kudos
2 More Replies
Labels