I am trying to experiment with remote_query() function according to the documentation. The feature is in public preview, so I assume it should be available to everyone now.select * from remote_query(
'my_connection',
database => 'mydb',
dbtable...
I am terraforming service principals in my Databricks workspace and it works great until I need to assign Git credentials to my SP. In the UI we have these options to configure credentials on service principal page:However the Terraform resource I fo...
My observations show that timestamp difference has type of INTERVAL DAY TO SECONDS:select typeof(getdate() - current_date())
-----------------------------------------
interval day to secondBut is it guaranteed? Can it be DAY TO MINUTE or, say, YEAR T...
I tried contact details on the bottom, but they seem to be generic Databricks contact and support links. The issue I faced was this:I think this word made its way to the stop list by a mistake.
I have a table, full scan of which takes ~20 minutes on my cluster. The table has "Time" TIMESTAMP column and "day" DATE column. The latter is computed (manually) as "Time" truncated to day and used for partitioning.I query the table using predicate ...
In Databricks, identities can be users, groups, and service principals. Cluster cannot be granted a permission. But the user of the cluster – yes, it has USE CONNECTION, and also the owner of connection.
I am experiencing a huge performance difference between shared and dedicated compute with spark.createDataFrame(pandas_df). Same code, same data, but it completes in 6 s on dedicated cluster and takes 6+ minutes on the shared cluster. >60 times diffe...
Yeah, and the most mind-bending things occur when you happen to create both account and workspace level group with the same name  In this case UI does not warn you in any way, but just shows the account group only. I once create a workspace group wi...
If you have a relatively modern Databricks instance, when you create a group in workspace UI, it creates an account-level group (which you can see in "Source" column – it says "Account"). So this process essentially consists of two steps: 1) create a...
