- 1218 Views
- 2 replies
- 0 kudos
Automatic schema rendering of files in unity catalog
Hi team,Can anyone please confirm if Unity catalog supports automatic schema rendering from csv, json, pdfs, and structured/unstructured files?Meaning, if i create a volume with path/location to folder (or S3 bucket) having such files, can unity cata...
- 1218 Views
- 2 replies
- 0 kudos
- 0 kudos
ok..thanks a lot @gchandra .So, I am new to Unity Catalog and particularly interested (and evaluating) the open sourced version of unity catalog (https://www.unitycatalog.io/)I know that, we can create volumes and those in turn can point to csv, json...
- 0 kudos
- 1872 Views
- 2 replies
- 1 kudos
Resolved! Error getting locations Unsupported response format: STREAM [Azure Databricks - Catalog Explorer]
Hi all,I saw this error when I checked my External Locations and Storage Credentials and used Catalog Explorer. The error message gives zero information for diagnosis. Do you have any idea what the reason is?Thank you.
- 1872 Views
- 2 replies
- 1 kudos
- 3931 Views
- 4 replies
- 1 kudos
How to disable spark connect in the databricks compute?
I want to be able to access the RDD methods of a Dataframe, but it seems that this is not supported in spark connect. I have been trying to disable spark connect in the spark config using,spark.databricks.service.server.enabled false but when I check...
- 3931 Views
- 4 replies
- 1 kudos
- 1 kudos
I have found that when the cluster is shared, it automatically uses that type of session, and in that case, I have not been able to disable it. I don't know if this is your situation. I have avoided some problems that I had with the previous clause.
- 1 kudos
- 7698 Views
- 1 replies
- 1 kudos
Cluster Upsize Issue: Storage Download Failure Slow
Hi,We're currently experiencing the following issue across our entire Databricks Workspace when either starting a cluster, running a workflow, or upscaling a running cluster. The following errors we receive on our AP clusters and job clusters are bel...
- 7698 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi @sdick_vg ,The error is about connectivity issues when trying to reach Azure Storage.Have you maybe enabled any kind of firewall in your organization recently?Could you run for example code to test DNS resolution to your storage account:Have you m...
- 1 kudos
- 2186 Views
- 2 replies
- 0 kudos
table deployment (DDL) from one catalog to another
HelloWe have a development, a test and a production environmentHow do you generally deploy DDL changes?So, alter a table in development and apply to test then productione.g.table1 has column1, column2, column3I add column4I now want to deploy this ch...
- 2186 Views
- 2 replies
- 0 kudos
- 0 kudos
Thanks.I'll step through this solution and see if I can get it working
- 0 kudos
- 1682 Views
- 1 replies
- 0 kudos
Testing and Issues Related to Admin Role Changes
Hello,I would like to ask a question regarding user permissions.Currently, all team members are admins. Recently, we plan to change the admin roles so that only I and another user, A, will be admins. The other members will retain general usage permis...
- 1682 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @Kaniz_Fatma , Can you please help me to delete my post as I accidentally resubmitted the post.Thank you.
- 0 kudos
- 1257 Views
- 1 replies
- 0 kudos
How to fully enable row-level concurrency on Databricks 14.1
Hey guys,I hope whoever's reading this is doing well.We're trying to enable row-level concurrency on Databricks 14.1. However, the documentation seems a bit contradictory as to whether this is possible, and whether the whole capability of row-level c...
- 1257 Views
- 1 replies
- 0 kudos
- 0 kudos
Yes, your understanding is correct. As best practice, use 14.3 LTS DBR, as LTS has longer support compared to 6 months.
- 0 kudos
- 929 Views
- 1 replies
- 0 kudos
Unable to publish notebook
Hello, I receive an error each time I attempt to publish my notebook. the error message does not provide any additional information on why it is occurring. I need to publish this for a bootcamp assignment, so please let me know what I can do to resol...
- 929 Views
- 1 replies
- 0 kudos
- 0 kudos
Try it from a different browser, or try it in incognito mode.
- 0 kudos
- 2037 Views
- 2 replies
- 0 kudos
Allow non-admin users to view the driver logs from a Unity Catalog-enabled pipeline
We are trying to enable the option to allow log reading to the non-admin users in the databricks wokspace but we are not able to see the obvious way to check them. The documentation is not showing after enabling the below property where to check the...
- 2037 Views
- 2 replies
- 0 kudos
- 4704 Views
- 3 replies
- 0 kudos
Grant permissions to groups on catalogs linked to the same metastore
Hi everyone!I am configuring several projects using Databricks, and I have a question regarding permission management in Unity Catalog. Here's the situation: Currently, I have two different Databricks resources in an Azure account, each with its resp...
- 4704 Views
- 3 replies
- 0 kudos
- 0 kudos
You could bind the catalog to specific workspaces, making them accessbile only from workspaces they are bound to.https://docs.databricks.com/en/catalogs/binding.htmlIn your example:if `my_catalog_2` is bound to `my_workspace_2` a user in `my_workspac...
- 0 kudos
- 1052 Views
- 1 replies
- 0 kudos
Databricks Free Trial - can I link it back from AWS to Azure?
I created a 14-day trial account on Databricks.com and linked it to my AWS. I have not used the trial yet and want to link it to Azure instead. Question: Can I do it, and if so, how?Remark: Please note that I have not used trial yet and the only rele...
- 1052 Views
- 1 replies
- 0 kudos
- 0 kudos
The easy way will be to create a new account in portal.azure.com and launch Azure Databricks with Pay as you go.
- 0 kudos
- 2518 Views
- 4 replies
- 1 kudos
set access control list for job cluster
Hello,currently, we are using an init script that calls the DBW API to add "can_attach_to" permissions for a specific group to initialize the job cluster. How can we set an access control list for job clusters? Is it possible to add it to a policy?
- 2518 Views
- 4 replies
- 1 kudos
- 1 kudos
Hi @mkieszek ,You can use REST API to set access control list for job clusters. The endpoint is for doing that you can find below/api/2.0/permissions/clusters/{cluster_id} (https://docs.databricks.com/api/workspace/clusters/setpermissions)And it's n...
- 1 kudos
- 3349 Views
- 5 replies
- 2 kudos
Resolved! Databricks bundle custom variables feature does not work for workspace host
Hi all,I am not sure whether this is a bug. I followed the document.My databricks.yml file: bundle: name: test variables: DATABRICKS_HOST: description: The host of the Databricks workspace. DATABRICKS_JOB_RUNNING_SERVICE_PRINCIPAL_CLIENT_I...
- 3349 Views
- 5 replies
- 2 kudos
- 2 kudos
Hi @AlbertWang ,The documentation states you cannot map workspace host:
- 2 kudos
- 1274 Views
- 1 replies
- 0 kudos
Installation of python module nevergrad with R package reticulate fails
I'm trying to run Robyn in Databricks but it fails since the python module nevergrad fails to install properly using the r package reticulate. This code still worked properly about 1,5 months ago so maybe the issue is on some new update in databricks...
- 1274 Views
- 1 replies
- 0 kudos
- 0 kudos
My same issue soleve this answer, thank you Kaniz!
- 0 kudos
- 4331 Views
- 5 replies
- 0 kudos
In Shared cluster with unity catalog Python process can't connect
The story is that I can access a service listening on port 80 using a single node cluster, but can't do the same using a shared node cluster. I read about the `spark.databricks.pyspark.iptable.outbound.whitelisted.ports` , however, setting that : `sp...
- 4331 Views
- 5 replies
- 0 kudos
- 0 kudos
@6502 you first need to allow the script in the metastore configuration (navgiate to catalog, on the top of the page there's a small cogwheel, click on your catalog name), then:
- 0 kudos
