cancel
Showing results for 
Search instead for 
Did you mean: 
Administration & Architecture
Explore discussions on Databricks administration, deployment strategies, and architectural best practices. Connect with administrators and architects to optimize your Databricks environment for performance, scalability, and security.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Priyam1
by New Contributor III
  • 4198 Views
  • 1 replies
  • 0 kudos

Access Logs

How can I check the timing when a particular AAD group was given access to a particular schema in a unity catalogue?Is there any API I can call to get this logs?

  • 4198 Views
  • 1 replies
  • 0 kudos
migq2
by New Contributor III
  • 5714 Views
  • 4 replies
  • 0 kudos

Use Unity External Location with full paths in delta_log

I have an external delta table in unity catalog (let's call it mycatalog.myschema.mytable) that only consists of a `_delta_log` directory that I create semi-manually, with the corresponding JSON files that define it. The JSON files point to parquet f...

  • 5714 Views
  • 4 replies
  • 0 kudos
Latest Reply
-werners-
Esteemed Contributor III
  • 0 kudos

I suggest you look at something else than UC for such cases.  I also wonder if delta lake is the right format.

  • 0 kudos
3 More Replies
rmubeenhsal
by New Contributor II
  • 7463 Views
  • 2 replies
  • 0 kudos

authorizationfailure on ls fs on mount point files

One of our users has as of last week started seeing an authorization failure when he tries to list the files in the Azure storage account using Databricks Cli or Databricks API(using Python). He can list files on the Databricks portal or through the ...

  • 7463 Views
  • 2 replies
  • 0 kudos
Latest Reply
Walter_C
Databricks Employee
  • 0 kudos

Have you checked the list of allowed ip addresses that are set for the Storage account in Azure? Is user using VPN or internal network, we might need to confirm if the network where the user is trying to list is set as allowed. 

  • 0 kudos
1 More Replies
curiousoctopus
by New Contributor III
  • 2748 Views
  • 1 replies
  • 0 kudos

User not authorised to copy files to dbfs

Hi,I'm trying to use a service principal to copy files to dbfs using the command line "databricks fs cp <source> <target>" but get back "User not authorised". I configured the authentication with PAT token and it is successful as I can deploy and lau...

  • 2748 Views
  • 1 replies
  • 0 kudos
Latest Reply
Walter_C
Databricks Employee
  • 0 kudos

In Databricks, data access permissions are often managed separately from workspace permissions. For DBFS, access control is typically managed through the underlying cloud storage (Azure Blob Storage, S3, etc.). The service principal needs to have the...

  • 0 kudos
Learnit
by New Contributor II
  • 3007 Views
  • 0 replies
  • 0 kudos

Delta Sharing resulting in Bad_Request

Hi All,Recipeint encountering an issue while trying to access my organizational data (providers data) in a Delta Sharing scenario(databricks to databricks), and I'm hoping to get some guidance on how to resolve it. Here is the error message recipient...

  • 3007 Views
  • 0 replies
  • 0 kudos
m997al
by Contributor III
  • 3209 Views
  • 0 replies
  • 0 kudos

Azure Databricks with standard private link (only one Databricks authentication workspace)?

We have successfully set up Azure Databricks with standard private link (front-end and back-end).The front-end uses the authentication workspace as prescribed in the documentation.Suppose we use a "custom DNS" for the configuration below.  Can we set...

m997al_0-1710778785048.png
  • 3209 Views
  • 0 replies
  • 0 kudos
ducng
by New Contributor II
  • 2665 Views
  • 1 replies
  • 0 kudos

Unity catalog metastore and databricks workspace provisioning

 Hi,I am using terraform to create a databricks metastore using this.Now the issue is that I want to use this metastore reference in another terraform project where I create the databricks workspaces but there is no data source for metastore. Can any...

Administration & Architecture
metastore
Terraform
  • 2665 Views
  • 1 replies
  • 0 kudos
Chaitanya07
by Databricks Partner
  • 4857 Views
  • 0 replies
  • 0 kudos

Databricks Rest APIs CORS Issue

Hello Team,We are currently integrating Databricks Rest APIs into our in-house application for managing access permissions. While testing with curl and Postman, we've successfully accessed certain APIs like listing cluster permission. However, we're ...

  • 4857 Views
  • 0 replies
  • 0 kudos
VJ3
by Contributor
  • 7161 Views
  • 0 replies
  • 0 kudos

Workflow (Job) Cluster Permission Management

Hello Team,I understand that as the Job Owner, they can grant additional permissions to other users to manage/run/view the job.If "Can Manage" permission is given to the other users, that user can Edit the job including the Run-As parameter to themse...

  • 7161 Views
  • 0 replies
  • 0 kudos
alm
by Databricks Partner
  • 2279 Views
  • 0 replies
  • 0 kudos

Cluster with shared access mode cannot query metastore

Hi, I have created a new UC enabled metastore using an Azure storage account and container connected to a Databricks workspace using an access connector. At first glance everything seems to work. I encounter a problem, however, when I try to query UC...

  • 2279 Views
  • 0 replies
  • 0 kudos
unauthenticated
by New Contributor
  • 6019 Views
  • 1 replies
  • 0 kudos

PyPI cluster libraries failing to get installed

Hi all,In my cluster, some of the PyPI cluster libraries started failing to get installed. It is weird because some of them get installed and some of are constantly failing. In every failed one, the error message is the same (just a package name is d...

  • 6019 Views
  • 1 replies
  • 0 kudos
Latest Reply
jacovangelder
Databricks MVP
  • 0 kudos

I've had this issue myself. What ended up to be the problem is I had windows line endings in my .sh script. You need to convert them to Linux line endings. 

  • 0 kudos
Kroy
by Contributor
  • 8345 Views
  • 4 replies
  • 1 kudos

Resolved! not able to see access schema inside system catalog

Hi,I am account admin role but not able to see access schema , I want to query audit log table     

Kroy_0-1710254944077.png
  • 8345 Views
  • 4 replies
  • 1 kudos
Latest Reply
Kroy
Contributor
  • 1 kudos

Done, needed to enable the system schema -- Run this command in notebook- to get list of available schema  %shcurl -v -X GET -H "Authorization: Bearer <PAT Token>" "https://adb-<xxx>.azuredatabricks.net/api/2.0/unity-catalog/metastores/<metastore-id>...

  • 1 kudos
3 More Replies
samini
by New Contributor II
  • 5106 Views
  • 2 replies
  • 0 kudos

Deleting MLflow runs programmatically

I would like to programmatically delete some MLflow runs based on a given run id. I am interested in the best practices on how to do this in Databricks workspaces.I know that MLflow cli has gc command which seems quite useful since it also deletes as...

  • 5106 Views
  • 2 replies
  • 0 kudos
Latest Reply
mameli
New Contributor II
  • 0 kudos

Do we have any updates on this? i'm using databricks and I have the same MlflowException

  • 0 kudos
1 More Replies