cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

geertvanhove
by New Contributor III
  • 11044 Views
  • 5 replies
  • 6 kudos

Resolved! connection to databricks data source in Visual Studio fails with SSL error

Hi,I'm new to databricks but am positively surprised by the product. We use databricks delta tables as source to build a tabular model, which will serve as data source for Power Bi. To develop our tabular model we use Visual studio to import tables ...

  • 11044 Views
  • 5 replies
  • 6 kudos
Latest Reply
Hubert-Dudek
Databricks MVP
  • 6 kudos

In Visual Studio do you use databricks-connect? Maybe it is enough to rerun "databricks-connect configure"

  • 6 kudos
4 More Replies
ivanychev
by Contributor II
  • 2919 Views
  • 3 replies
  • 3 kudos

AWS + Cluster with custom docker image = export metrics to CloudWatch?

Is there a working setup on setting up metrics export to CloudWatch while using custom docker images for cluster creation? I've tried to set up the CloudWatch agent manually, but launching `amzon-cloudwatch-agent-ctl` in the bootstrap script fails wi...

  • 2919 Views
  • 3 replies
  • 3 kudos
Latest Reply
Atanu
Databricks Employee
  • 3 kudos

We do not support gangila with custom docker too. but let me cross verify if we are supporting cloudwatch for the same. Sorry for the inconvenience @Sergey Ivanychev​ 

  • 3 kudos
2 More Replies
apw
by New Contributor II
  • 4603 Views
  • 1 replies
  • 2 kudos

Arrow R package fails to install

# Databricks notebook source .libPaths()   # COMMAND ----------   dir("/databricks/spark/R/lib")   # COMMAND ----------   ## Add current working directory to library paths .libPaths(c(getwd(), .libPaths()))   # COMMAND ----------   ## The latest vers...

Arrow Fail Message" data-fileid="0698Y00000JFZosQAH
  • 4603 Views
  • 1 replies
  • 2 kudos
Latest Reply
Atanu
Databricks Employee
  • 2 kudos

@Anthony McGrath​ can you please download and upload to DBFS and see if the issue still persists?You can check if any global initscript is reinstalling this to your cluster.

  • 2 kudos
Martin1
by New Contributor II
  • 4494 Views
  • 2 replies
  • 2 kudos

Notebook metadata

HelloI would like to view metadata about the notebooks in the Workspace folder hierarchy, for example date created, modified, by user, etc.Is this possible?

  • 4494 Views
  • 2 replies
  • 2 kudos
Latest Reply
Atanu
Databricks Employee
  • 2 kudos

@Martin Aronsson​, you need to follow the notebook revision history (https://docs.databricks.com/notebooks/notebooks-use.html#revision-history) I believe. Also you can try https://docs.microsoft.com/en-us/azure/databricks/administration-guide/account...

  • 2 kudos
1 More Replies
User16835756816
by Databricks Employee
  • 6413 Views
  • 1 replies
  • 8 kudos

Announcing: Workflows!

Databricks is excited to announce the general availability of Databricks Workflows to you, our community. Databricks Workflows is the fully managed lakehouse orchestration service for all your teams to build reliable data, analytics, and AI workflow...

  • 6413 Views
  • 1 replies
  • 8 kudos
Latest Reply
PawanShukla
New Contributor III
  • 8 kudos

I am trying to run the Workflow Pipeline with smaple code shared in getting start.. and getting the below error :DataPlaneException: Failed to start the DLT service on cluster 0526-084319-7hucy1np. Please check the stack trace below or driver logs fo...

  • 8 kudos
as999
by New Contributor III
  • 3177 Views
  • 2 replies
  • 3 kudos

Terraform import multiple notebook copy from repo's?

From below article, i am able to copy only single notebook to dbrick workspace and it's not supporting to copy the multiple notebook using asterisks i.e * and also under resource databrick_notebook, for_each statement is not recognizingdatabricks_n...

  • 3177 Views
  • 2 replies
  • 3 kudos
Latest Reply
Atanu
Databricks Employee
  • 3 kudos

Hi @as999​ is there any error you are getting or it's just simply not copying multiple notebook, can you please share your code too so that I can take a look.Thanks.

  • 3 kudos
1 More Replies
Sophia_Ars
by New Contributor II
  • 2061 Views
  • 1 replies
  • 1 kudos

Abrupt Subscription Cancellation Issues

Hello Community,I've got informed from Help desk to post this issue in community.We've contacted all supportive entities: billing team, help desk and sales team,but the issue hasn't solved yet.My team(Ars Praxia) has issue of sudden cancellation of s...

  • 2061 Views
  • 1 replies
  • 1 kudos
CHANDY
by Databricks Partner
  • 1504 Views
  • 0 replies
  • 0 kudos

real time data processing

Say I am getting a customer record from an website. I want to read the massage & then insert/update that one to snowflake table , depending on the records insert/update is successful I need to respond back the success / failure massage in say 1 sec. ...

  • 1504 Views
  • 0 replies
  • 0 kudos
Sunny
by New Contributor III
  • 1105 Views
  • 0 replies
  • 1 kudos

Integrate exe into workflow

We need to execute a long running exe running on a windows machine and thinking of ways to integrate with the workflow. The plan is to include the exe as a task in the Databricks workflow.​​We are thinking of couple of approachesCreate a DB table and...

  • 1105 Views
  • 0 replies
  • 1 kudos
timothy_uk
by New Contributor III
  • 4060 Views
  • 2 replies
  • 4 kudos

Resolved! Optimum Standard & Premium Tier Strategy

Hi,I would like to deploy Databricks workspaces to build a delta lakehouse to server both scheduled jobs/processing and ad-hoc/analytical querying workloads. Databricks users comprise of both data engineers and data analysts. In terms of requirements...

  • 4060 Views
  • 2 replies
  • 4 kudos
Latest Reply
timothy_uk
New Contributor III
  • 4 kudos

Hi all thank you for informative answers!

  • 4 kudos
1 More Replies
edwardh
by New Contributor III
  • 6912 Views
  • 5 replies
  • 6 kudos

How to call Cloud Fetch APIs?

About Cloud Fetch mentioned in this article:https://databricks.com/blog/2021/08/11/how-we-achieved-high-bandwidth-connectivity-with-bi-tools.htmlAre there any public APIs that can be called directly without ODBC or JDBC drivers? Thanks.

  • 6912 Views
  • 5 replies
  • 6 kudos
Latest Reply
edwardh
New Contributor III
  • 6 kudos

Hi @Kaniz Fatma​, can you please give some help on this question? Thanks

  • 6 kudos
4 More Replies
Deepak_Bhutada
by Databricks Employee
  • 3881 Views
  • 3 replies
  • 3 kudos

Retrieve workspace instance name on E2 architecture (multi-tenant) in notebook running on job cluster

I have a databricks job on E2 architecture in which I want to retrieve the workspace instance name within a notebook running in a Job cluster context so that I can use it further in my use case. While the call dbutils.notebook.entry_point.getDbutils(...

  • 3881 Views
  • 3 replies
  • 3 kudos
Latest Reply
Thomas_B_
New Contributor II
  • 3 kudos

Found workaround for Azure Databricks question above: dbutils.notebook.getContext().apiUrl will return the regional URI, but this forwards to the workspace-specific one if the workspace id is specified with o=.

  • 3 kudos
2 More Replies
Phani1
by Databricks MVP
  • 2713 Views
  • 1 replies
  • 2 kudos

Resolved! is it possible to have multiple tabs in Dashboard? if not is there any workaround for this.

is it possible to have multiple tabs in Dashboard? if not is there any workaround for this.

  • 2713 Views
  • 1 replies
  • 2 kudos
Latest Reply
Prabakar
Databricks Employee
  • 2 kudos

I don't think it will be possible. However, you can raise a feature request via our ideas portal with the requirements so that it might be considered in the future.https://docs.databricks.com/resources/ideas.html

  • 2 kudos
Labels