cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

junaid
by New Contributor II
  • 1541 Views
  • 0 replies
  • 0 kudos

Issue in creating workspace - Custom AWS Configuration

Hello,We have tried to created new Databricks Workspace using "Custom AWS Configuration" and we have given our own VPC (Customer managed VPC) and tried but workspace failed to launch. There are Documents Available for resolve this error , According t...

  • 1541 Views
  • 0 replies
  • 0 kudos
sriramkumar
by New Contributor II
  • 4952 Views
  • 3 replies
  • 0 kudos

New Databricks Driver gives SQLNonTransientConnectionException when trying to connect to Databricks Instance

import com.databricks.client.jdbc.DataSource;   import java.sql.*;   public class testDatabricks { public static void main(String[] args) throws SQLException { String dbUrl = "jdbc:databricks://<hostname>:443;HttpPath=<HttpPath>;"; // Cop...

  • 4952 Views
  • 3 replies
  • 0 kudos
Latest Reply
Atanu
Databricks Employee
  • 0 kudos

This looks like due to maintenance on US . Are you still facing the issue @Sriramkumar Thamizharasan​ Is your workspace on eastus and eastus2 ?

  • 0 kudos
2 More Replies
repcak
by New Contributor III
  • 8341 Views
  • 4 replies
  • 3 kudos

Resolved! Delta Live Tables with EventHub

Hello,I would like to integrate Databricks Delta Live Tables with Eventhub, but i cannot install com.microsoft.azure:azure-eventhubs-spark_2.12:2.3.17 on delta live cluster.I tried installed in using Init script (by adding it in Json cluster settings...

image
  • 8341 Views
  • 4 replies
  • 3 kudos
Latest Reply
Atanu
Databricks Employee
  • 3 kudos

I think this has some details https://docs.microsoft.com/en-us/azure/event-hubs/event-hubs-kafka-spark-tutorial @Kacper Mucha​ is the issue resolved ?

  • 3 kudos
3 More Replies
Soma
by Valued Contributor
  • 5039 Views
  • 5 replies
  • 6 kudos

Resolved! Start Up Notebook in Databricks

Hi Team,Is it possible to have a startup notebook in databricks similar to init sql script ( This need to run on start of every cluster )

  • 5039 Views
  • 5 replies
  • 6 kudos
Latest Reply
Soma
Valued Contributor
  • 6 kudos

@Kaniz Fatma​ sure will do

  • 6 kudos
4 More Replies
Soma
by Valued Contributor
  • 4210 Views
  • 0 replies
  • 0 kudos

Spark Failure Error Unable to download spark docker Image

Cluster terminated. Reason: Spark Image Download Failure  "reason": { "code": "SPARK_IMAGE_DOWNLOAD_FAILURE", "type": "SERVICE_FAULT", "parameters": { "instance_id": "6565aa39b0ae4fe69c7fe6f313e3ca2a", "databricks_error_message": "Failed to set up th...

  • 4210 Views
  • 0 replies
  • 0 kudos
venkad
by Contributor
  • 2509 Views
  • 1 replies
  • 2 kudos

Resolved! Is Databricks Light Runtime Discontinued?

The last Databricks Light runtime release was 2.4 Extended Support. There was no Light version for Spark 3.x. Is Databricks Light runtime discontinued? If not, when we can expect the next DBR Light version?

  • 2509 Views
  • 1 replies
  • 2 kudos
Latest Reply
Anonymous
Not applicable
  • 2 kudos

Hi @Venkadeshwaran K​, I looked around, and it does look like there won't be future light runtimes. We can't hire enough engineers to maintain and develop everything, and light is one of the casualties of that.

  • 2 kudos
geertvanhove
by New Contributor III
  • 10731 Views
  • 5 replies
  • 6 kudos

Resolved! connection to databricks data source in Visual Studio fails with SSL error

Hi,I'm new to databricks but am positively surprised by the product. We use databricks delta tables as source to build a tabular model, which will serve as data source for Power Bi. To develop our tabular model we use Visual studio to import tables ...

  • 10731 Views
  • 5 replies
  • 6 kudos
Latest Reply
Hubert-Dudek
Databricks MVP
  • 6 kudos

In Visual Studio do you use databricks-connect? Maybe it is enough to rerun "databricks-connect configure"

  • 6 kudos
4 More Replies
ivanychev
by Contributor II
  • 2782 Views
  • 3 replies
  • 3 kudos

AWS + Cluster with custom docker image = export metrics to CloudWatch?

Is there a working setup on setting up metrics export to CloudWatch while using custom docker images for cluster creation? I've tried to set up the CloudWatch agent manually, but launching `amzon-cloudwatch-agent-ctl` in the bootstrap script fails wi...

  • 2782 Views
  • 3 replies
  • 3 kudos
Latest Reply
Atanu
Databricks Employee
  • 3 kudos

We do not support gangila with custom docker too. but let me cross verify if we are supporting cloudwatch for the same. Sorry for the inconvenience @Sergey Ivanychev​ 

  • 3 kudos
2 More Replies
apw
by New Contributor II
  • 4182 Views
  • 1 replies
  • 2 kudos

Arrow R package fails to install

# Databricks notebook source .libPaths()   # COMMAND ----------   dir("/databricks/spark/R/lib")   # COMMAND ----------   ## Add current working directory to library paths .libPaths(c(getwd(), .libPaths()))   # COMMAND ----------   ## The latest vers...

Arrow Fail Message&quot; data-fileid=&quot;0698Y00000JFZosQAH
  • 4182 Views
  • 1 replies
  • 2 kudos
Latest Reply
Atanu
Databricks Employee
  • 2 kudos

@Anthony McGrath​ can you please download and upload to DBFS and see if the issue still persists?You can check if any global initscript is reinstalling this to your cluster.

  • 2 kudos
Martin1
by New Contributor II
  • 4293 Views
  • 2 replies
  • 2 kudos

Notebook metadata

HelloI would like to view metadata about the notebooks in the Workspace folder hierarchy, for example date created, modified, by user, etc.Is this possible?

  • 4293 Views
  • 2 replies
  • 2 kudos
Latest Reply
Atanu
Databricks Employee
  • 2 kudos

@Martin Aronsson​, you need to follow the notebook revision history (https://docs.databricks.com/notebooks/notebooks-use.html#revision-history) I believe. Also you can try https://docs.microsoft.com/en-us/azure/databricks/administration-guide/account...

  • 2 kudos
1 More Replies
User16835756816
by Databricks Employee
  • 6188 Views
  • 1 replies
  • 8 kudos

Announcing: Workflows!

Databricks is excited to announce the general availability of Databricks Workflows to you, our community. Databricks Workflows is the fully managed lakehouse orchestration service for all your teams to build reliable data, analytics, and AI workflow...

  • 6188 Views
  • 1 replies
  • 8 kudos
Latest Reply
PawanShukla
New Contributor III
  • 8 kudos

I am trying to run the Workflow Pipeline with smaple code shared in getting start.. and getting the below error :DataPlaneException: Failed to start the DLT service on cluster 0526-084319-7hucy1np. Please check the stack trace below or driver logs fo...

  • 8 kudos
as999
by New Contributor III
  • 3044 Views
  • 2 replies
  • 3 kudos

Terraform import multiple notebook copy from repo's?

From below article, i am able to copy only single notebook to dbrick workspace and it's not supporting to copy the multiple notebook using asterisks i.e * and also under resource databrick_notebook, for_each statement is not recognizingdatabricks_n...

  • 3044 Views
  • 2 replies
  • 3 kudos
Latest Reply
Atanu
Databricks Employee
  • 3 kudos

Hi @as999​ is there any error you are getting or it's just simply not copying multiple notebook, can you please share your code too so that I can take a look.Thanks.

  • 3 kudos
1 More Replies
Sophia_Ars
by New Contributor II
  • 1946 Views
  • 1 replies
  • 1 kudos

Abrupt Subscription Cancellation Issues

Hello Community,I've got informed from Help desk to post this issue in community.We've contacted all supportive entities: billing team, help desk and sales team,but the issue hasn't solved yet.My team(Ars Praxia) has issue of sudden cancellation of s...

  • 1946 Views
  • 1 replies
  • 1 kudos
Labels