Databricks Workspace import api size limitation

[pavansharma36]

As libraries from dbfs is deprecated and support is going to be removed. We are moving libs from dbfs to workspace files.

But while uploading libraries using https://docs.databricks.com/api/azure/workspace/workspace/import api there seems to be limit on size of content which is 10mb. When I try from UI to upload lib I can upload with size more than 10mb but not from api.

Not able to find any other way to upload files using api, can anyone share details how to get this done with api.

Thanks

[Giri-Patcham · Accepted solution]

@pavansharma36  You can try volumes and use Files API to upload 
https://docs.databricks.com/api/azure/workspace/files

[pavansharma36]

PUT <DATABRICKS_HOST>/api/2.0/fs/directories?directory_path=/Workspace/Shared/iw_cgu1prrv/test

gives 

{
    "error_code": "BAD_REQUEST",
    "message": "Invalid path:",
    "details": [
        {
            "@type": "type.googleapis.com/google.rpc.ErrorInfo",
            "reason": "FILES_API_INVALID_PATH",
            "domain": "filesystem.databricks.com",
            "metadata": {
                "validation_error": ""
            }
        }
    ]
}

Tried multiple values

eg. /Volumes/samples/default/test

[Giri-Patcham]

@pavansharma36 Try something like this 

PUT <DATABRICKS_HOST>/api/2.0/fs/directories/Volumes/samples/default/test

[pavansharma36]

Looks like volumes not supported for Standard tier.

{
    "error_code": "BAD_REQUEST",
    "message": "Unity Catalog is not available for feature tier STANDARD_TIER.",
    "details": [
        {
            "@type": "type.googleapis.com/google.rpc.ErrorInfo",
            "reason": "FILES_API_MANAGED_CATALOG_FEATURE_DISABLED",
            "domain": "filesystem.databricks.com",
            "metadata": {
                "unity_catalog_error_message": "Unity Catalog is not available for feature tier STANDARD_TIER."
            }
        }
    ]
}

Not able to create Workspace directories from files api

Whats point of exposing files api onto standard tier when its not usable.

[NaraKris_40883]

Can we use this API to upload files to /workspace/ location? Any sample curl request ? I am using 

-X PUT https://<HOST_NAME>/api/2.0/fs/files/Workspace/Shared/jars/all.jar and getting 

{

  "error_code" : "BAD_REQUEST",

  "message" : "Invalid path:",

  "details" : [ {

    "@type" : "type.googleapis.com/google.rpc.ErrorInfo",

    "reason" : "FILES_API_INVALID_PATH",

    "domain" : "filesystem.databricks.com",

    "metadata" : {

      "validation_error" : ""

    }

  } ]