Databricks has SCIM support at both the workspace and account levels. With Unity Catalog you should be using the account level so you may ignore any workspace level SCIM instructions. The documentation will be slightly different for each cloud.AWS:...
I can't say for sure what your exact limitation is going to be but two things I have come across in EMR bake offs are:1) bottleneck in the network throughput. Verify the S3 reads and writes are happening at similar throughput rates. This is a cloud...
In the step 3 you will want to consider using Databricks Workflows for orchestration. The ADF databricks notebook activity is not actively developed by microsoft and the API it uses is legacy by Databricks So neither vendor is actively supporting t...
