cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Help
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Start a conversation
GC-James
by • Contributor II
  • 17936 Views
  • 8 replies
  • 10 kudos

RserveException: eval failed

Sometimes when I am running R code in a Databricks notebook I am given this error. The cell I am running fails, and my whole R 'session' seems to get screwed up. For example my stored variables disappear, and I have to re-load my packages etc. It is ...

rserve_error
Data Engineering
Reply
  • 17936 Views
  • 8 replies
  • 10 kudos
View Replies
Latest Reply
data_warrior
New Contributor III
  • 10 kudos

The error file is attached here.

  • 10 kudos
7 More Replies
Braxx
by • Contributor II
  • 8130 Views
  • 2 replies
  • 1 kudos

Resolved! delta table storage

I couldn't find it clearly explained anywhere, so hope sb here shed some light on that.Few questions:1) Where does delta tables are stored? Docs say: "Delta Lake uses versioned Parquet files to store your data in your cloud storage"So where exactly i...

Data Engineering
Reply
  • 8130 Views
  • 2 replies
  • 1 kudos
View Replies
Latest Reply
Braxx
Contributor II
  • 1 kudos

thanks, very helpful

  • 1 kudos
1 More Replies
sebg
by • Databricks Partner
  • 3722 Views
  • 0 replies
  • 1 kudos

Using (python) import on azure databricks

Hello,My team is currently working on azure databricks with a mid sized repo. When we wish to import pyspark functions and classes from other notebooks we currently use %run <relpath>which is less than ideal.I would like to replicate the functionalit...

image
Data Engineering
Reply
  • 3722 Views
  • 0 replies
  • 1 kudos
MariusC
by • New Contributor III
  • 15565 Views
  • 5 replies
  • 5 kudos

Resolved! Power BI with Databricks SQL Endpoint

Hello,We are trying to load a Delta table from an Azure Data Lake Storage container into Power BI using the Databricks SQL Endpoint.We configured the SQL Workspace data to have access to the ADLS Delta table and created a view; we are able to query t...

error screenshot sample_error sample_query ADLS_delta_query
Data Engineering
Reply
  • 15565 Views
  • 5 replies
  • 5 kudos
View Replies
Latest Reply
Atanu
Databricks Employee
  • 5 kudos

@Marius Condescu​ Could you please include below spark config and try-spark.hadoop.fs.azure.account.oauth.provider.type.ariaprime.dfs.core.windows.net org.apache.hadoop.fs.azurebfs.oauth2.ClientCredsTokenProviderspark.hadoop.fs.azure.account.auth.typ...

  • 5 kudos
4 More Replies
Anonymous
by • Not applicable
  • 1966 Views
  • 0 replies
  • 0 kudos

Data Visualized | Lego style

Something fun for your Friday! If you are a visual person like me, you may like this image that was recently shared in our internal Databricks slack instance. Who else 李s Legos? If you have seen data all 6 ways with Databricks, give this a 李 !!!

Image
Data Engineering
Reply
  • 1966 Views
  • 0 replies
  • 0 kudos
Raymond_Garcia
by • Contributor II
  • 2013 Views
  • 0 replies
  • 1 kudos

EsHadoopNoNodesLeftException: Connection error (check network and/or proxy settings)- all nodes failed; tried [[elasticsearch_server:80]]

Hi, I wondered if some of you have had this issue before and how it can be solved. In a Databricks Job, we have a UBQ with a Painless script for ES. these are the options. Staging and prod are the same configurations, but Staging is failing with the ...

Data Engineering
Reply
  • 2013 Views
  • 0 replies
  • 1 kudos
alexa
by • New Contributor III
  • 9717 Views
  • 2 replies
  • 0 kudos

Resolved! How to use dateadd in databricks sql ?

I am trying to something like this but getting error like :Error in SQL statement: AnalysisException: Undefined function: 'DATEADD'. This function is neither a registered temporary function nor a permanent function registered in the database 'default...

Data Engineering
Reply
  • 9717 Views
  • 2 replies
  • 0 kudos
View Replies
Latest Reply
SergeRielau
Databricks Employee
  • 0 kudos

Dateadd was added in DBR 10.4 and is in DBSQL current.SELECT DATEADD(HOUR,IFNULL(100, 0),current_date) AS Date_Created_Local=> 2022-05-31T04:00:00.000+0000.You can also use one of these casts to turn any wellformed string into an interval:SELECT curr...

  • 0 kudos
1 More Replies
thushar
by • Databricks Partner
  • 8432 Views
  • 0 replies
  • 0 kudos

How to install wheel package from git repo

Using VS code for development and a wheel package is created for shipment.We put this wheel package in Azure data lake storage and ADB notebook accessed this wheel package and installed it in the cluster. It is working fine. But instead of keeping th...

image.png
Data Engineering
Reply
  • 8432 Views
  • 0 replies
  • 0 kudos
junaid
by • New Contributor II
  • 1604 Views
  • 0 replies
  • 0 kudos

Issue in creating workspace - Custom AWS Configuration

Hello,We have tried to created new Databricks Workspace using "Custom AWS Configuration" and we have given our own VPC (Customer managed VPC) and tried but workspace failed to launch. There are Documents Available for resolve this error , According t...

Data Engineering
Reply
  • 1604 Views
  • 0 replies
  • 0 kudos
sriramkumar
by • New Contributor II
  • 5129 Views
  • 3 replies
  • 0 kudos

New Databricks Driver gives SQLNonTransientConnectionException when trying to connect to Databricks Instance

import com.databricks.client.jdbc.DataSource;   import java.sql.*;   public class testDatabricks { public static void main(String[] args) throws SQLException { String dbUrl = "jdbc:databricks://<hostname>:443;HttpPath=<HttpPath>;"; // Cop...

Data Engineering
Reply
  • 5129 Views
  • 3 replies
  • 0 kudos
View Replies
Latest Reply
Atanu
Databricks Employee
  • 0 kudos

This looks like due to maintenance on US . Are you still facing the issue @Sriramkumar Thamizharasan​ Is your workspace on eastus and eastus2 ?

  • 0 kudos
2 More Replies
repcak
by • New Contributor III
  • 8594 Views
  • 4 replies
  • 3 kudos

Resolved! Delta Live Tables with EventHub

Hello,I would like to integrate Databricks Delta Live Tables with Eventhub, but i cannot install com.microsoft.azure:azure-eventhubs-spark_2.12:2.3.17 on delta live cluster.I tried installed in using Init script (by adding it in Json cluster settings...

image
Data Engineering
Reply
  • 8594 Views
  • 4 replies
  • 3 kudos
View Replies
Latest Reply
Atanu
Databricks Employee
  • 3 kudos

I think this has some details https://docs.microsoft.com/en-us/azure/event-hubs/event-hubs-kafka-spark-tutorial @Kacper Mucha​ is the issue resolved ?

  • 3 kudos
3 More Replies
Labels