cancel
Showing results for 
Search instead for 
Did you mean: 
Administration & Architecture
Explore discussions on Databricks administration, deployment strategies, and architectural best practices. Connect with administrators and architects to optimize your Databricks environment for performance, scalability, and security.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

karthik_p
by Esteemed Contributor
  • 1690 Views
  • 3 replies
  • 0 kudos

Disaster Recovery Issue

We are trying to create Disaster Recovery for UC enabled Workspaces in Azure. our UC metastore are in different regions.1. we are trying to use Deep Clone2. In source we are adding region2 metastore as external location3. able to do deep cloneproblem...

  • 1690 Views
  • 3 replies
  • 0 kudos
Latest Reply
-werners-
Esteemed Contributor III
  • 0 kudos

Right I get it.Actually cloning it as external seems logical to me, for the moment, as unity cannot manage the other metastore.For the moment I would go with cloning the data and then creating an external table of that.Not ideal, but at least you hav...

  • 0 kudos
2 More Replies
JDL
by New Contributor III
  • 2962 Views
  • 1 replies
  • 1 kudos

Tie Parquet files in Azure ADLS to Databricks table

Hello All,I have databricks delta table with files residing in Azure Data Lake. I understand, when I load create table and load data from databricks, it creates respective folder and files for table in ADLS. I am wondering if there is reverse way to ...

  • 2962 Views
  • 1 replies
  • 1 kudos
Latest Reply
JDL
New Contributor III
  • 1 kudos

Thanks @Retired_mod this is helpful

  • 1 kudos
RonMoody
by New Contributor II
  • 7118 Views
  • 1 replies
  • 0 kudos

service principal table accesses not showing up in system.audit

When we run jobs using service principals system.audit doesn't show any table accesses (getTable). Volume (getVolume) shows up for service principals. Same query when run as a user shows up in system.audit. I know system.audit is in public preview. W...

  • 7118 Views
  • 1 replies
  • 0 kudos
Latest Reply
RonMoody
New Contributor II
  • 0 kudos

hi @Retired_mod thanks so much for your reply ! I was referring to https://docs.databricks.com/en/administration-guide/system-tables/audit-logs.html which is part of databricks core offering and isn't related to ServiceNow's offering. I am assuming t...

  • 0 kudos
cltj
by New Contributor III
  • 2020 Views
  • 1 replies
  • 1 kudos

Three level name space naming standard

Hi all, I have not been successful in getting a good grip of the naming conventions for the three level name space. Initially i learned about bronze, silver and gold, but i am confused where to put this. The obvious choice may be to use the {catalog}...

  • 2020 Views
  • 1 replies
  • 1 kudos
aayusha3
by New Contributor II
  • 2908 Views
  • 4 replies
  • 2 kudos

Internal error: Attach your notebook to a different compute or restart the current compute.

I am currently using a personal computer cluster [13.3 LTS (includes Apache Spark 3.4.1, Scala 2.12)] on GCP attached to a notebook. After running a few command lines without an issue, I end up getting this error  Internal error. Attach your notebook...

aayusha3_0-1700411861325.png
  • 2908 Views
  • 4 replies
  • 2 kudos
Latest Reply
amandaolens
New Contributor III
  • 2 kudos

@Martin74  same here martin.

  • 2 kudos
3 More Replies
amandaolens
by New Contributor III
  • 5890 Views
  • 3 replies
  • 2 kudos

Internal error. Attach your notebook to a different compute or restart the current compute. java.lan

Internal error. Attach your notebook to a different compute or restart the current compute.java.lang.RuntimeException: abort: DriverClient destroyed at com.databricks.backend.daemon.driver.DriverClient.$anonfun$poll$3(DriverClient.scala:577) at scala...

  • 5890 Views
  • 3 replies
  • 2 kudos
Latest Reply
amandaolens
New Contributor III
  • 2 kudos

@Retired_mod yeah, in one dataset there are slightly higher data points.schema are same.when the spark crashed i have checked the memory usage.it was around 50%.

  • 2 kudos
2 More Replies
niklas
by Contributor
  • 3568 Views
  • 1 replies
  • 0 kudos

Resolved! Error: cannot create metastore data access

I'm in the progress of enabling Databricks Unity Catalog and encountered a problem with the databricks_metastore_data_access Terraform resource:resource "databricks_metastore_data_access" "this" { provider = databricks.account-level metastore_id...

Administration & Architecture
metastore
Terraform
Unity Catalog
  • 3568 Views
  • 1 replies
  • 0 kudos
Latest Reply
niklas
Contributor
  • 0 kudos

Found a solution. Please see my answer on Stack Overflow:https://stackoverflow.com/questions/77440091/databricks-unity-catalog-error-cannot-create-metastore-data-access/77506306#77506306

  • 0 kudos
NateJ
by New Contributor II
  • 3316 Views
  • 4 replies
  • 2 kudos

Failed to start cluster: Large docker image

I have a large Docker image in our AWS ECR repo. The image is 27.4 GB locally and 11539.79 MB compressed in ECR.The error from the Event Log is:Failed to add 2 containers to the compute. Will attempt retry: true. Reason: Docker image pull failureJSON...

  • 3316 Views
  • 4 replies
  • 2 kudos
Latest Reply
amoghjain
New Contributor II
  • 2 kudos

I have a similar problem. a 10gb image pulls fine but a 31gb image doesnt. both workers and drivers have 64gb memory. i get the timeout error with "Cannot launch the cluster because pulling the docker image failed. Please double check connectivity fr...

  • 2 kudos
3 More Replies
Learnit
by New Contributor II
  • 2292 Views
  • 0 replies
  • 0 kudos

Databricks deployment and automation tools comparison.

Hello All, As a newcomer to databricks, I am seeking guidance on automation within databricks environments. What are the best best practices for deployment, and how do Terraform, the REST API, and the databricks SDK compare in terms of advantages and...

  • 2292 Views
  • 0 replies
  • 0 kudos
MasJamei
by New Contributor II
  • 1359 Views
  • 1 replies
  • 0 kudos

Notebook Id level of uniqueness

Hi there,We know that notebook ids are unique. https://docs.databricks.com/en/workspace/workspace-details.html but I want to know in what level they're unique. For example, if Notebook Ids are unique within a workspace, or are they universally unique...

  • 1359 Views
  • 1 replies
  • 0 kudos

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group
Labels