Inspecting underlying HTTP traffic while using Databricks UI to import files in Workspace, it turns out (as expected) Databricks API is used, with requests similar to:

So, @Naveenkumar1811 use Databricks API with SP identity in a similar way as expected above. Docs on API are here: https://docs.databricks.com/api/azure/workspace/workspace/import . I guess, limitation on size is not in the API side, but inside Databricks CLI .