Databricks Community

If your goal is to charge back other teams or business units based on consumption then you should enforce tags on all clusters/compute. These tags will show up on your Azure bill which you can then identify which groups used which resources.

Databricks Light is the open source Apache Spark runtime and does not come with any type of client for Delta Lake pre-installed. You'll need to manually install open source Delta Lake in order to do any reads or writes.See our docs and release notes ...

I think you are looking for the INSERT OVERWRITE command in Spark SQL. Check out the documentation here: https://docs.databricks.com/spark/latest/spark-sql/language-manual/sql-ref-syntax-dml-insert-overwrite-table.html

Yeah - here is the link with details on how to integrate with Tabelu's different products https://docs.databricks.com/integrations/bi/tableau.html

Yes of course, notebooks are not exclusively tied to a specific cluster so you can pick any available/visible cluster to attach the notebook on when you want to run it.Also please keep in mind that by doing this half-way through executing a notebook,...

In the UI, Databricks will list the running endpoints on top. Programmatically you can get information about the endpoints using the REST APIs. You will likely need to use a combo of the list endpoint to get all the endpoints. The for each endpoint u...

Yes, you can use the SCIM API integration to provision both users and groups. We have examples for Okta, Azure AD and OneLogin, but any SCIM-enabled IdP should suffice.

It does not. You will be able to retain past runs and view them for up to 60 days regardless if you updating the configuration of the job.

Yes you can. Databricks maintains a history of your job runs for up to 60 days. If you need to preserve job runs, Databricks recommends that you export results before they expire. For more information, see https://docs.databricks.com/jobs.html#export...

How about updates and inserts? does it refresh automatically?