In Spark 4.0, there are no more data type mismatches when converting dynamic JSONs, as the new data type VariantType comes with a new function to parse JSONs. Stay tuned for 4.0 release.
SQL serverless now use cache even after termination thanks to the remote cache. You can benefit from 1 min auto-termination when still utilizing cache benefits.
If you want to copy files in Azure, ADF is usually the fastest option (for example TB of csvs, parquets). If you want to copy tables, just use CLONE. If it is files with code just use Repos and branches.
Remember that keys are for information purposes (they don't validate data integrity). They are used for information in a few places (Feature tables, online tables, PowerBi modelling). The best is to define them in CREATE TABLE syntax, for example:CRE...
but serverless is in Databricks Subscription so you need to whitelist traffic from there.Please check that list. Storage firewall needs to allow connections https://learn.microsoft.com/en-us/azure/databricks/security/network/serverless-network-securi...
