cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Help
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Start a conversation
as999
by • New Contributor III
  • 13990 Views
  • 8 replies
  • 6 kudos

Databrick hive metastore location?

In databrick, where is hive metastore location is it control plane or data plane? for prod systems In terms of security what preventions should be taken to secure hive metastore?

Data Engineering
Reply
  • 13990 Views
  • 8 replies
  • 6 kudos
View Replies
Latest Reply
Prabakar
Databricks Employee
  • 6 kudos

@as999​ The default metastore is managed by Databricks. If you are concerned about security and would like to have your own metastore you can go for the external metastore setup. You have the details steps in the below doc for setting up the external...

  • 6 kudos
7 More Replies
Trey
by • New Contributor III
  • 3185 Views
  • 2 replies
  • 6 kudos

Resolved! Is it a good idea to use a managed delta table as a temporal table?

Hi all!I would like to use a managed delta table as a temporal table, meaning:to create a managed table in the middle of ETL processto drop the managed table right after the processThis way I can perform merge, insert, or delete oprations better than...

Data Engineering
Reply
  • 3185 Views
  • 2 replies
  • 6 kudos
View Replies
Latest Reply
karthik_p
Esteemed Contributor
  • 6 kudos

@Kwangwon Yi​ Instead of performance, main issue with managed table is whenever you delete table, data under that table gets deleted.If you have good use case on Reporting, best approach is to go with external storage location to store your managed t...

  • 6 kudos
1 More Replies
jwilliam
by • Contributor
  • 2043 Views
  • 2 replies
  • 2 kudos

Resolved! Does libraries installation happen on Data Plane or Control Plane?

Currently, when I install libraries on my clusters. This errors happens:WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLEOFError(8, 'EOF occurred in violation of protocol...

Data Engineering
Reply
  • 2043 Views
  • 2 replies
  • 2 kudos
View Replies
Latest Reply
Sivaprasad1
Valued Contributor II
  • 2 kudos

@John William​ : Yeah that's true. All the clusters will be residing in the data plane.

  • 2 kudos
1 More Replies
aladda
by • Databricks Employee
  • 4075 Views
  • 1 replies
  • 0 kudos

Resolved! Where does Databricks store its Notebooks? Are they on a file system in the control plane or RDS/data management system of some kind

Data Engineering
Reply
  • 4075 Views
  • 1 replies
  • 0 kudos
View Replies
Latest Reply
aladda
Databricks Employee
  • 0 kudos

Notebooks in Databricks are part of the WebApp which is run & managed by databricks from the Control Plane. See the high level architecture here for details - https://docs.databricks.com/getting-started/overview.html

  • 0 kudos
Labels