Databricks

aladda · 06-25-2021

aladda · 06-25-2021

Databricks must have access to at least two subnets for each workspace, with each subnet in a different availability zone per docs here

aladda · 06-23-2021

aladda · 06-23-2021

It is best to avoid collecting stats on long strings. You typically want to collect stats on column that are used in filter, where clauses, joins and on which you tend to performance aggregations - typically numerical valuesYou can avoid collecting s...

aladda · 06-23-2021

aladda · 06-25-2021

You could potentially do this through a Global Init Script - https://docs.databricks.com/clusters/init-scripts.html

aladda · 06-25-2021

Can you provide some additional details on this? What components are we comparing the states for?

aladda · 06-25-2021

Deleting the Delta log directory would cause you to lose the underlying transaction history on the delta table and other delta related optimizations. In effect the table would be converted to a Parquet table at that point

aladda · 06-25-2021

Global: run on every cluster in the workspace. They can help you to enforce consistent cluster configurations across your workspace. Use them carefully because they can cause unanticipated impacts, like library conflicts. Only admin users can create ...

aladda · 06-25-2021

See this knowledge base article for details - https://kb.databricks.com/clusters/overwrite-log4j-logs.html

Databricks

User Stats

User Activity

Why does databricks need two IP addresses per node in a cluster

Why do Databricks deployments require 2 subnets for each workspace

How are stats collected on a Delta column utilized

What are the recommendations around collecting stats on long strings in a Delta Table

How many columns does Delta Engine collect stats on for a Delta Table

Re: Are there some shard-level setting to make every cluster in that shard use KMS encryption when writing to S3

Re: What are the advantages of using RocksDB State store compared to HDFS backed state store

Re: What to do if I accidentally deleted Delta log directory

Re: What is the advantage of using the new Global init script compared to the legacy init script

Re: Can I use a custom log4j file for the Databricks cluster