Databricks Community

The issue is that the python file saved under DBFS not as a workspace notebook. When you given /artifacts/client-state vector/0.0.0/bootstrap.py, the workspace will search the notebook(python file in this case) under the folder that under Workspace t...

Yes, you can set up an instance profile that can access the S3 bucket and then only give certain users privilege to use the instance profile. For more details, you can check here

No, I do not believe that is possible. However, I would be interested in understanding a use case where that is ideal behavior. How Does Schema Enforcement Work?Delta Lake uses schema validation on write, which means that all new writes to a table ar...

You can get a static IP at the workspace level https://docs.microsoft.com/en-us/azure/databricks/kb/cloud/azure-vnet-single-ip

Open a notebook and run the command dbutils.fs.rm("/FileStore/tables/your_table_name.csv") referencing this link https://docs.databricks.com/data/databricks-file-system.html

This is worked for me. Thank you @acorson​ 

A metaclass in Python is a class of a class that defines how a class behaves. A class is itself an instance of a metaclass. A class in Python defines how the instance of the class will behave.

Leave it turned on. the bet is with each Spark version released AQE will get better and better and eventually will lead to a much more performance optimisation plan than manually trying to tune it.

Cartesian product is the worst type of join that you should always avoid, it tends to product N x M where N & M are the left and right table cardinality. unless you specifically want to use it, then you should avoid and opt for faster joins (inner, l...

You need to get the REST service API access tokens and make sure the Databricks VPC (or VNET on Azure) have connectivity to the VPC where this REST API service resides.

If the data in your table is huge, try to combine OPTIMIZE with WHERE so you only perform OPTIMIZE on a subset of the data rather than all data. see documentation here.

The OPTIMIZE ZORDER operation now uses Hilbert space-filling curves by default. This approach provides better clustering characteristics than Z-order in higher dimensions. For Delta tables using OPTIMIZE ZORDER with many columns, Hilbert curves can s...

You can view them in the workspace Home Page --> Top Right Click on User Settings --> Access Tokens.Or you can use the List API to get them.