- 939 Views
- 2 replies
- 0 kudos
Automatic schema rendering of files in unity catalog
Hi team,Can anyone please confirm if Unity catalog supports automatic schema rendering from csv, json, pdfs, and structured/unstructured files?Meaning, if i create a volume with path/location to folder (or S3 bucket) having such files, can unity cata...
- 939 Views
- 2 replies
- 0 kudos
- 0 kudos
ok..thanks a lot @gchandra .So, I am new to Unity Catalog and particularly interested (and evaluating) the open sourced version of unity catalog (https://www.unitycatalog.io/)I know that, we can create volumes and those in turn can point to csv, json...
- 0 kudos
- 1565 Views
- 2 replies
- 1 kudos
Resolved! Error getting locations Unsupported response format: STREAM [Azure Databricks - Catalog Explorer]
Hi all,I saw this error when I checked my External Locations and Storage Credentials and used Catalog Explorer. The error message gives zero information for diagnosis. Do you have any idea what the reason is?Thank you.
- 1565 Views
- 2 replies
- 1 kudos
- 3017 Views
- 4 replies
- 1 kudos
How to disable spark connect in the databricks compute?
I want to be able to access the RDD methods of a Dataframe, but it seems that this is not supported in spark connect. I have been trying to disable spark connect in the spark config using,spark.databricks.service.server.enabled false but when I check...
- 3017 Views
- 4 replies
- 1 kudos
- 1 kudos
I have found that when the cluster is shared, it automatically uses that type of session, and in that case, I have not been able to disable it. I don't know if this is your situation. I have avoided some problems that I had with the previous clause.
- 1 kudos
- 3846 Views
- 5 replies
- 2 kudos
Resolved! How to create Storage Credential using Service Principal [Azure]
As the document indicates, An Azure Databricks access connector is a first-party Azure resource that lets you connect managed identities to an Azure Databricks account. You must have the Contributor role or higher on the access connector resource in ...
- 3846 Views
- 5 replies
- 2 kudos
- 2 kudos
Thank you, @szymon_dybczak. This is what I thought. After deploying the Databricks workspace, it automatically creates the Databricks managed `Access Connector for Azure Databricks` in the Databricks managed resource group.As I understand, I should c...
- 2 kudos
- 4937 Views
- 1 replies
- 1 kudos
Cluster Upsize Issue: Storage Download Failure Slow
Hi,We're currently experiencing the following issue across our entire Databricks Workspace when either starting a cluster, running a workflow, or upscaling a running cluster. The following errors we receive on our AP clusters and job clusters are bel...
- 4937 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi @sdick_vg ,The error is about connectivity issues when trying to reach Azure Storage.Have you maybe enabled any kind of firewall in your organization recently?Could you run for example code to test DNS resolution to your storage account:Have you m...
- 1 kudos
- 1574 Views
- 1 replies
- 0 kudos
Error: PERMISSION_DENIED: AWS IAM role does
Hello, We are trying to setup a new workspace. However we are getting following error. Workspace failed to launch.Error: PERMISSION_DENIED: AWS IAM role does not have READ permissions on url s3://jk-databricks-prods3/unity-catalog/742920957025975.Pl...
- 1574 Views
- 1 replies
- 0 kudos
- 0 kudos
Hey! I'm experiencing this with the latest Terraform release. Try 1.51.0 if you are deploying via TF, downgrading fixed this for me.
- 0 kudos
- 1626 Views
- 2 replies
- 0 kudos
table deployment (DDL) from one catalog to another
HelloWe have a development, a test and a production environmentHow do you generally deploy DDL changes?So, alter a table in development and apply to test then productione.g.table1 has column1, column2, column3I add column4I now want to deploy this ch...
- 1626 Views
- 2 replies
- 0 kudos
- 0 kudos
Thanks.I'll step through this solution and see if I can get it working
- 0 kudos
- 1408 Views
- 1 replies
- 0 kudos
Testing and Issues Related to Admin Role Changes
Hello,I would like to ask a question regarding user permissions.Currently, all team members are admins. Recently, we plan to change the admin roles so that only I and another user, A, will be admins. The other members will retain general usage permis...
- 1408 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @Kaniz_Fatma , Can you please help me to delete my post as I accidentally resubmitted the post.Thank you.
- 0 kudos
- 947 Views
- 1 replies
- 0 kudos
How to fully enable row-level concurrency on Databricks 14.1
Hey guys,I hope whoever's reading this is doing well.We're trying to enable row-level concurrency on Databricks 14.1. However, the documentation seems a bit contradictory as to whether this is possible, and whether the whole capability of row-level c...
- 947 Views
- 1 replies
- 0 kudos
- 0 kudos
Yes, your understanding is correct. As best practice, use 14.3 LTS DBR, as LTS has longer support compared to 6 months.
- 0 kudos
- 664 Views
- 1 replies
- 0 kudos
Unable to publish notebook
Hello, I receive an error each time I attempt to publish my notebook. the error message does not provide any additional information on why it is occurring. I need to publish this for a bootcamp assignment, so please let me know what I can do to resol...
- 664 Views
- 1 replies
- 0 kudos
- 0 kudos
Try it from a different browser, or try it in incognito mode.
- 0 kudos
- 1436 Views
- 2 replies
- 0 kudos
Allow non-admin users to view the driver logs from a Unity Catalog-enabled pipeline
We are trying to enable the option to allow log reading to the non-admin users in the databricks wokspace but we are not able to see the obvious way to check them. The documentation is not showing after enabling the below property where to check the...
- 1436 Views
- 2 replies
- 0 kudos
- 3264 Views
- 3 replies
- 0 kudos
Grant permissions to groups on catalogs linked to the same metastore
Hi everyone!I am configuring several projects using Databricks, and I have a question regarding permission management in Unity Catalog. Here's the situation: Currently, I have two different Databricks resources in an Azure account, each with its resp...
- 3264 Views
- 3 replies
- 0 kudos
- 0 kudos
You could bind the catalog to specific workspaces, making them accessbile only from workspaces they are bound to.https://docs.databricks.com/en/catalogs/binding.htmlIn your example:if `my_catalog_2` is bound to `my_workspace_2` a user in `my_workspac...
- 0 kudos
- 801 Views
- 1 replies
- 0 kudos
Databricks Free Trial - can I link it back from AWS to Azure?
I created a 14-day trial account on Databricks.com and linked it to my AWS. I have not used the trial yet and want to link it to Azure instead. Question: Can I do it, and if so, how?Remark: Please note that I have not used trial yet and the only rele...
- 801 Views
- 1 replies
- 0 kudos
- 0 kudos
The easy way will be to create a new account in portal.azure.com and launch Azure Databricks with Pay as you go.
- 0 kudos
- 1971 Views
- 4 replies
- 1 kudos
set access control list for job cluster
Hello,currently, we are using an init script that calls the DBW API to add "can_attach_to" permissions for a specific group to initialize the job cluster. How can we set an access control list for job clusters? Is it possible to add it to a policy?
- 1971 Views
- 4 replies
- 1 kudos
- 1 kudos
Hi @mkieszek ,You can use REST API to set access control list for job clusters. The endpoint is for doing that you can find below/api/2.0/permissions/clusters/{cluster_id} (https://docs.databricks.com/api/workspace/clusters/setpermissions)And it's n...
- 1 kudos
- 2069 Views
- 5 replies
- 2 kudos
Resolved! Databricks bundle custom variables feature does not work for workspace host
Hi all,I am not sure whether this is a bug. I followed the document.My databricks.yml file: bundle: name: test variables: DATABRICKS_HOST: description: The host of the Databricks workspace. DATABRICKS_JOB_RUNNING_SERVICE_PRINCIPAL_CLIENT_I...
- 2069 Views
- 5 replies
- 2 kudos
- 2 kudos
Hi @AlbertWang ,The documentation states you cannot map workspace host:
- 2 kudos
