cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Anonymous
by Not applicable
  • 1346 Views
  • 0 replies
  • 0 kudos

Data Visualized | Lego style

Something fun for your Friday! If you are a visual person like me, you may like this image that was recently shared in our internal Databricks slack instance. Who else 李s Legos? If you have seen data all 6 ways with Databricks, give this a 李 !!!

Image
  • 1346 Views
  • 0 replies
  • 0 kudos
Raymond_Garcia
by Contributor II
  • 1282 Views
  • 0 replies
  • 1 kudos

EsHadoopNoNodesLeftException: Connection error (check network and/or proxy settings)- all nodes failed; tried [[elasticsearch_server:80]]

Hi, I wondered if some of you have had this issue before and how it can be solved. In a Databricks Job, we have a UBQ with a Painless script for ES. these are the options. Staging and prod are the same configurations, but Staging is failing with the ...

  • 1282 Views
  • 0 replies
  • 1 kudos
alexa
by New Contributor III
  • 6661 Views
  • 2 replies
  • 0 kudos

Resolved! How to use dateadd in databricks sql ?

I am trying to something like this but getting error like :Error in SQL statement: AnalysisException: Undefined function: 'DATEADD'. This function is neither a registered temporary function nor a permanent function registered in the database 'default...

  • 6661 Views
  • 2 replies
  • 0 kudos
Latest Reply
SergeRielau
Databricks Employee
  • 0 kudos

Dateadd was added in DBR 10.4 and is in DBSQL current.SELECT DATEADD(HOUR,IFNULL(100, 0),current_date) AS Date_Created_Local=> 2022-05-31T04:00:00.000+0000.You can also use one of these casts to turn any wellformed string into an interval:SELECT curr...

  • 0 kudos
1 More Replies
thushar
by Contributor
  • 6413 Views
  • 0 replies
  • 0 kudos

How to install wheel package from git repo

Using VS code for development and a wheel package is created for shipment.We put this wheel package in Azure data lake storage and ADB notebook accessed this wheel package and installed it in the cluster. It is working fine. But instead of keeping th...

image.png
  • 6413 Views
  • 0 replies
  • 0 kudos
junaid
by New Contributor II
  • 1154 Views
  • 0 replies
  • 0 kudos

Issue in creating workspace - Custom AWS Configuration

Hello,We have tried to created new Databricks Workspace using "Custom AWS Configuration" and we have given our own VPC (Customer managed VPC) and tried but workspace failed to launch. There are Documents Available for resolve this error , According t...

  • 1154 Views
  • 0 replies
  • 0 kudos
sriramkumar
by New Contributor II
  • 3221 Views
  • 3 replies
  • 0 kudos

New Databricks Driver gives SQLNonTransientConnectionException when trying to connect to Databricks Instance

import com.databricks.client.jdbc.DataSource;   import java.sql.*;   public class testDatabricks { public static void main(String[] args) throws SQLException { String dbUrl = "jdbc:databricks://<hostname>:443;HttpPath=<HttpPath>;"; // Cop...

  • 3221 Views
  • 3 replies
  • 0 kudos
Latest Reply
Atanu
Databricks Employee
  • 0 kudos

This looks like due to maintenance on US . Are you still facing the issue @Sriramkumar Thamizharasan​ Is your workspace on eastus and eastus2 ?

  • 0 kudos
2 More Replies
repcak
by New Contributor III
  • 5483 Views
  • 4 replies
  • 3 kudos

Resolved! Delta Live Tables with EventHub

Hello,I would like to integrate Databricks Delta Live Tables with Eventhub, but i cannot install com.microsoft.azure:azure-eventhubs-spark_2.12:2.3.17 on delta live cluster.I tried installed in using Init script (by adding it in Json cluster settings...

image
  • 5483 Views
  • 4 replies
  • 3 kudos
Latest Reply
Atanu
Databricks Employee
  • 3 kudos

I think this has some details https://docs.microsoft.com/en-us/azure/event-hubs/event-hubs-kafka-spark-tutorial @Kacper Mucha​ is the issue resolved ?

  • 3 kudos
3 More Replies
Soma
by Valued Contributor
  • 3445 Views
  • 5 replies
  • 6 kudos

Resolved! Start Up Notebook in Databricks

Hi Team,Is it possible to have a startup notebook in databricks similar to init sql script ( This need to run on start of every cluster )

  • 3445 Views
  • 5 replies
  • 6 kudos
Latest Reply
Soma
Valued Contributor
  • 6 kudos

@Kaniz Fatma​ sure will do

  • 6 kudos
4 More Replies
Soma
by Valued Contributor
  • 3567 Views
  • 0 replies
  • 0 kudos

Spark Failure Error Unable to download spark docker Image

Cluster terminated. Reason: Spark Image Download Failure  "reason": { "code": "SPARK_IMAGE_DOWNLOAD_FAILURE", "type": "SERVICE_FAULT", "parameters": { "instance_id": "6565aa39b0ae4fe69c7fe6f313e3ca2a", "databricks_error_message": "Failed to set up th...

  • 3567 Views
  • 0 replies
  • 0 kudos
venkad
by Contributor
  • 1759 Views
  • 1 replies
  • 2 kudos

Resolved! Is Databricks Light Runtime Discontinued?

The last Databricks Light runtime release was 2.4 Extended Support. There was no Light version for Spark 3.x. Is Databricks Light runtime discontinued? If not, when we can expect the next DBR Light version?

  • 1759 Views
  • 1 replies
  • 2 kudos
Latest Reply
Anonymous
Not applicable
  • 2 kudos

Hi @Venkadeshwaran K​, I looked around, and it does look like there won't be future light runtimes. We can't hire enough engineers to maintain and develop everything, and light is one of the casualties of that.

  • 2 kudos

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group
Labels