Databricks Community

for any other non-private previews, they can check out admin console --> advanced tab. there are tons of toggles there to enable/disable features. if it’s not there, there usually isn’t an easy (or direct) way of disabling

you can go to spark UI and Kill it

This will be coming with Unity. There will be a third-level namespace. The default HM will be the default database in unity most likely.

Is it really %%bash or may be you are referring to %sh or %fs or dbutils command

Scala Use JVM to run its code, Scala cannot run different applications at a time with complete isolation of each task inside single jvm , that is the reason Scala doesn't support high concurrency cluster, I don't think it is on road map

We can read from Solr as Dataframe and SolrRDD as well (each RDD is a SolrDocument)more details available here

That’s only available at Premium and Enterprise SKUs in AWS.See the "Enterprise Security" section here:https://databricks.com/product/aws-pricing

On AWS: Enable bucket versioningOn Azure: Enable Soft DeletesOf course this needs to be done before you actually start writing the data.

Please raise a feature request via ideas portal for XML support in autoloader As a workaround, you could look at reading this with wholeTextFiles (which loads the data into a PairRDD with one record per input file) and parsing it with from_xml from ...

Here is the link to the configuration properties https://docs.databricks.com/dev-tools/databricks-connect.html#step-2-configure-connection-properties

You can set the configurations on the Databricks cluster UIhttps://docs.databricks.com/clusters/configure.html#spark-configurationTo see the default configuration, run the below code in a notebook:%sql set;

Notebooks are not stored in DBFS. They cannot be directly listed from the file system. You should use the Databricks REST API to list and get the detailshttps://docs.databricks.com/dev-tools/api/latest/workspace.html#list

To time travel to a particular version, it's necessary to have the JSON file for that particular version. the JSON files in the delta_log have default retention of 30 days. So by default, we can time travel only up to 30 days. The retention of the D...

The important factors deciding partition columns are:Even distribution of data. Choose the column that is commonly or widely accessed or queried. Do not create multiple levels of partition, as you can end up with a large number of small files.

If the table is external the files are not deleted. For managed table, the underlying files get deleted. Essentially a "DROP TABLE" command is submitted under the hood.