cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Raymond_Garcia
by Contributor II
  • 822 Views
  • 0 replies
  • 1 kudos

EsHadoopNoNodesLeftException: Connection error (check network and/or proxy settings)- all nodes failed; tried [[elasticsearch_server:80]]

Hi, I wondered if some of you have had this issue before and how it can be solved. In a Databricks Job, we have a UBQ with a Painless script for ES. these are the options. Staging and prod are the same configurations, but Staging is failing with the ...

  • 822 Views
  • 0 replies
  • 1 kudos
alexa
by New Contributor III
  • 4651 Views
  • 2 replies
  • 0 kudos

Resolved! How to use dateadd in databricks sql ?

I am trying to something like this but getting error like :Error in SQL statement: AnalysisException: Undefined function: 'DATEADD'. This function is neither a registered temporary function nor a permanent function registered in the database 'default...

  • 4651 Views
  • 2 replies
  • 0 kudos
Latest Reply
SergeRielau
Valued Contributor
  • 0 kudos

Dateadd was added in DBR 10.4 and is in DBSQL current.SELECT DATEADD(HOUR,IFNULL(100, 0),current_date) AS Date_Created_Local=> 2022-05-31T04:00:00.000+0000.You can also use one of these casts to turn any wellformed string into an interval:SELECT curr...

  • 0 kudos
1 More Replies
alejandrofm
by Valued Contributor
  • 582 Views
  • 2 replies
  • 0 kudos

How to run sparkStream for earlier (not future messages)

Hi, I'm listening to a stream for kinesis, don't need the data in real-time, so I could run it on an hourly basis looking to achieve two things:-Save money by don't have a cluster up 24/7-Have bigger files saved for each readThe stream is constant so...

  • 582 Views
  • 2 replies
  • 0 kudos
Latest Reply
Kaniz
Community Manager
  • 0 kudos

Hi @Alejandro Martinez​ , Here is a somewhat similar issue on S.O. Please look and let us know if that helped.

  • 0 kudos
1 More Replies
sriramkumar
by New Contributor II
  • 1876 Views
  • 3 replies
  • 0 kudos

New Databricks Driver gives SQLNonTransientConnectionException when trying to connect to Databricks Instance

import com.databricks.client.jdbc.DataSource;   import java.sql.*;   public class testDatabricks { public static void main(String[] args) throws SQLException { String dbUrl = "jdbc:databricks://<hostname>:443;HttpPath=<HttpPath>;"; // Cop...

  • 1876 Views
  • 3 replies
  • 0 kudos
Latest Reply
Atanu
Esteemed Contributor
  • 0 kudos

This looks like due to maintenance on US . Are you still facing the issue @Sriramkumar Thamizharasan​ Is your workspace on eastus and eastus2 ?

  • 0 kudos
2 More Replies
repcak
by New Contributor III
  • 2871 Views
  • 6 replies
  • 3 kudos

Resolved! Delta Live Tables with EventHub

Hello,I would like to integrate Databricks Delta Live Tables with Eventhub, but i cannot install com.microsoft.azure:azure-eventhubs-spark_2.12:2.3.17 on delta live cluster.I tried installed in using Init script (by adding it in Json cluster settings...

image
  • 2871 Views
  • 6 replies
  • 3 kudos
Latest Reply
Atanu
Esteemed Contributor
  • 3 kudos

I think this has some details https://docs.microsoft.com/en-us/azure/event-hubs/event-hubs-kafka-spark-tutorial @Kacper Mucha​ is the issue resolved ?

  • 3 kudos
5 More Replies
Soma
by Valued Contributor
  • 1586 Views
  • 8 replies
  • 6 kudos

Resolved! Start Up Notebook in Databricks

Hi Team,Is it possible to have a startup notebook in databricks similar to init sql script ( This need to run on start of every cluster )

  • 1586 Views
  • 8 replies
  • 6 kudos
Latest Reply
Soma
Valued Contributor
  • 6 kudos

@Kaniz Fatma​ sure will do

  • 6 kudos
7 More Replies
venkad
by Contributor
  • 827 Views
  • 2 replies
  • 2 kudos

Resolved! Is Databricks Light Runtime Discontinued?

The last Databricks Light runtime release was 2.4 Extended Support. There was no Light version for Spark 3.x. Is Databricks Light runtime discontinued? If not, when we can expect the next DBR Light version?

  • 827 Views
  • 2 replies
  • 2 kudos
Latest Reply
Anonymous
Not applicable
  • 2 kudos

Hi @Venkadeshwaran K​, I looked around, and it does look like there won't be future light runtimes. We can't hire enough engineers to maintain and develop everything, and light is one of the casualties of that.

  • 2 kudos
1 More Replies
geertvanhove
by New Contributor III
  • 5334 Views
  • 7 replies
  • 6 kudos

Resolved! connection to databricks data source in Visual Studio fails with SSL error

Hi,I'm new to databricks but am positively surprised by the product. We use databricks delta tables as source to build a tabular model, which will serve as data source for Power Bi. To develop our tabular model we use Visual studio to import tables ...

  • 5334 Views
  • 7 replies
  • 6 kudos
Latest Reply
Kaniz
Community Manager
  • 6 kudos

Hi @geert vanhove​ â€‹ , Just a friendly follow-up. Do you still need help, or have you resolved your problem with the above solutions? Please let us know.

  • 6 kudos
6 More Replies
ivanychev
by Contributor
  • 883 Views
  • 3 replies
  • 3 kudos

AWS + Cluster with custom docker image = export metrics to CloudWatch?

Is there a working setup on setting up metrics export to CloudWatch while using custom docker images for cluster creation? I've tried to set up the CloudWatch agent manually, but launching `amzon-cloudwatch-agent-ctl` in the bootstrap script fails wi...

  • 883 Views
  • 3 replies
  • 3 kudos
Latest Reply
Atanu
Esteemed Contributor
  • 3 kudos

We do not support gangila with custom docker too. but let me cross verify if we are supporting cloudwatch for the same. Sorry for the inconvenience @Sergey Ivanychev​ 

  • 3 kudos
2 More Replies
Sophia_Ars
by New Contributor II
  • 666 Views
  • 3 replies
  • 2 kudos

Abrupt Subscription Cancellation Issues

Hello Community,I've got informed from Help desk to post this issue in community.We've contacted all supportive entities: billing team, help desk and sales team,but the issue hasn't solved yet.My team(Ars Praxia) has issue of sudden cancellation of s...

  • 666 Views
  • 3 replies
  • 2 kudos
Latest Reply
Kaniz
Community Manager
  • 2 kudos

Hi @Jayeon Jang​ , Thank you for reaching out!I understand how frustrating this must have been for you.We value our customers’ time, and this should not have happened.I appreciate you making us aware of your negative experience.I will relay this mess...

  • 2 kudos
2 More Replies
Sunny
by New Contributor III
  • 289 Views
  • 0 replies
  • 1 kudos

Integrate exe into workflow

We need to execute a long running exe running on a windows machine and thinking of ways to integrate with the workflow. The plan is to include the exe as a task in the Databricks workflow.​​We are thinking of couple of approachesCreate a DB table and...

  • 289 Views
  • 0 replies
  • 1 kudos
Labels
Top Kudoed Authors