cancel
Turn on suggestions
Showing results for 
Search instead for 
Did you mean: 
Help
Administration & Architecture
Explore discussions on Databricks administration, deployment strategies, and architectural best practices. Connect with administrators and architects to optimize your Databricks environment for performance, scalability, and security.
cancel
Turn on suggestions
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Start a conversation
karthiknuvepro
by • New Contributor II
  • 141 Views
  • 1 replies
  • 0 kudos

GCP Databricks | Workspace Creation Error: Storage Credentials Limit Reached

Hi Team,We are encountering an issue while trying to create a Databricks Workspace in the GCP region us-central1. Below is the error message:Error Message:Workspace Status: FailedDetails: Workspace failed to launch.Error: BAD REQUEST: Cannot create 1...

Administration & Architecture
Reply
  • 141 Views
  • 1 replies
  • 0 kudos
View Replies
Latest Reply
Alberto_Umana
Databricks Employee
  • 0 kudos

Hi @karthiknuvepro, Do you have an active support plan? Over a ticket with us we can request the increase of this limit.

  • 0 kudos
rdadhichi
by • New Contributor II
  • 294 Views
  • 2 replies
  • 0 kudos

Disable 'Allow trusted Microsoft services to bypass this firewall' for Azure Key Vault

Currently even when using vnet injected Databricks workspace, we are unable to fetch the secrets from AKV if the 'Allow trusted Microsoft services to bypass this firewall' is disabled.The secret is used a AKV backed secret scope and the key vault is ...

Administration & Architecture
Reply
  • 294 Views
  • 2 replies
  • 0 kudos
View Replies
Latest Reply
Alberto_Umana
Databricks Employee
  • 0 kudos

Hi @rdadhichi, Have you set "Allow access from" to "Private endpoint and selected networks" on the firewall?

  • 0 kudos
1 More Replies
AnkitShah
by • New Contributor II
  • 240 Views
  • 4 replies
  • 0 kudos

How do we get user list who accessed/downloaded specific model in Unity catalog for last 6 months

How do we get user list who accessed/downloaded specific model in Unity catalog for last 6 months

Administration & Architecture
Reply
  • 240 Views
  • 4 replies
  • 0 kudos
View Replies
Latest Reply
Alberto_Umana
Databricks Employee
  • 0 kudos

Hi @AnkitShah, I just tried on my end and found these 2 tables that might be useful. They do not exact show who downloaded a model artifact but who interacted with it: https://docs.databricks.com/en/ai-gateway/configure-ai-gateway-endpoints.html#usag...

  • 0 kudos
3 More Replies
RicksDB
by • Contributor III
  • 452 Views
  • 6 replies
  • 0 kudos

Governance to restrict compute creation

Hi,Cluster policies used to be an easy way to handle governance on computes. However, more and more, there seem to be no way to control many new compute features within the platform. We currently have this issue for model serving endpoints and vector...

Administration & Architecture
Reply
  • 452 Views
  • 6 replies
  • 0 kudos
View Replies
Latest Reply
nskiran
New Contributor III
  • 0 kudos

If you are looking to restrict end users to create certain cluster configuration only, you can do so by using databricks APIs. Through python and Databricks API, you can specify what kind of cluster configurations are allowed and also restrict users ...

  • 0 kudos
5 More Replies
dbuserng
by • New Contributor II
  • 406 Views
  • 1 replies
  • 4 kudos

High memory usage on Databricks cluster

In my team we have a very high memory usage even when the cluster has just been started and nothing has been run yet. Additionally, memory usage never drops to lower levels - total used memory always fluctuates around 14GB.Where is this memory usage ...

Administration & Architecture
Reply
  • 406 Views
  • 1 replies
  • 4 kudos
View Replies
Latest Reply
-werners-
Esteemed Contributor III
  • 4 kudos

This is not necessarily an issue.  Linux uses a lot of RAM for caching but this does not mean it cannot be released for processes (dynamic memory mgmt).Basically the philosophy is that RAM that is not used (so actually 'free') is useless.Here is a re...

  • 4 kudos
jimbender
by • New Contributor II
  • 158 Views
  • 1 replies
  • 0 kudos

Newbie DAB question regarding wheels

I am trying to build a wheel using a DAB.  It errors saying I don't have permissions to install my wheel onto a cluster I am have been given.  Is it possible to just upload the wheel to a subdir the /Shared directory and use it from there instead of ...

Administration & Architecture
Reply
  • 158 Views
  • 1 replies
  • 0 kudos
View Replies
Latest Reply
Walter_C
Databricks Employee
  • 0 kudos

May I know the exact error you are getting on the cluster?You can use the following code to use a wheel in a shared folder: resources: jobs: my-job: name: my-job tasks: - task_key: my-task new_cluster: ...

  • 0 kudos
sparrap
by • New Contributor
  • 711 Views
  • 2 replies
  • 0 kudos

Error when Connecting Databricks Cluster to RStudio Desktop App

Hi! I am trying to connect RStudio to my Databricks Cluster, I already change the permissions to CAN MANAGE and CAN ATTACH to the cluster. Also I have verified to have the correct python version and Databricks version in my computer.This is the error...

Administration & Architecture
Reply
  • 711 Views
  • 2 replies
  • 0 kudos
View Replies
Latest Reply
mikvaar
New Contributor II
  • 0 kudos

This seems to solve the problem: https://github.com/sparklyr/sparklyr/issues/3449Apparently sparklyr requires that Unity Catalog is enabled on the cluster in order to get the connection working right.

  • 0 kudos
1 More Replies
RameshRetnasamy
by • Contributor
  • 13492 Views
  • 25 replies
  • 23 kudos

Resolved! Unable to login to Azure Databricks Account Console

I have a personal Azure pay-as-you-go subscription in which I have the 'Global Administrator' role. I am also the databricks account administrator.Until two weeks ago, I was able to access the databricks account console without any issues, but I am f...

Screenshot 2024-08-07 at 12.13.53.png Screenshot 2024-08-07 at 12.15.30.png Screenshot 2024-08-07 at 12.17.20.png Screenshot 2024-08-07 at 12.18.35.png
Administration & Architecture
account-console
Databricks
Reply
  • 13492 Views
  • 25 replies
  • 23 kudos
View Replies
Latest Reply
satyaki_guha
New Contributor II
  • 23 kudos

Hi @RameshRetnasamy thanks for your solution.I was able to login to Databricks Account Console but was unable to set the metastore path.Something got broke along the way.It would definitely be beneficial if we could have the same email address for ou...

  • 23 kudos
24 More Replies
cmathieu
by • New Contributor II
  • 230 Views
  • 1 replies
  • 0 kudos

Multiple feature branches per user using Databricks Asset Bundles

I'm currently helping a team migrating to DABs from dbx and they would like to be able to work on multiple features at the same time.What I was able to do is pass the current branch as a variable in the root_path and various names, so when the bundle...

Administration & Architecture
Reply
  • 230 Views
  • 1 replies
  • 0 kudos
View Replies
Latest Reply
cmathieu
New Contributor II
  • 0 kudos

deleting the DABs in Databricks and also in my local version under .databricks solved this one for me. 

  • 0 kudos
sparkplug
by • New Contributor III
  • 275 Views
  • 1 replies
  • 1 kudos

Resolved! How do I track notebooks in all purpose compute?

I am trying to map out costs for a Shared cluster used in our organization. Since Databricks does not store the sessions in all purpose compute or who accessed the cluster, what are some possible options that I can track which notebooks were attached...

Administration & Architecture
Reply
  • 275 Views
  • 1 replies
  • 1 kudos
View Replies
Latest Reply
Alberto_Umana
Databricks Employee
  • 1 kudos

Hi @sparkplug, You can use the audit logs and billing usage table: https://docs.databricks.com/en/admin/account-settings/audit-logs.html

  • 1 kudos
navi_bricks
by • New Contributor II
  • 1157 Views
  • 9 replies
  • 1 kudos

Need to move files from one Volume to other

We recently enabled Unity catalog on our workspace, as part of certain transformations(Custom clustered Datapipelines(python)) we need to move file from one volume to other volume. As the job itself runs on a service principal that has access to exte...

Administration & Architecture
Reply
  • 1157 Views
  • 9 replies
  • 1 kudos
View Replies
Latest Reply
Dnirmania
Contributor
  • 1 kudos

Not all job clusters work well with Volumes. I used following type cluster to access files from Volume. 

  • 1 kudos
8 More Replies
bdanielatl
by • New Contributor II
  • 229 Views
  • 1 replies
  • 3 kudos

Resolved! Markdown Cells Do Not Render Consistently

When I am creating a notebook in the UI editor on DataBricks, markdown cells do not always render after I run them. They still appear in 'editing mode'. See the screenshot below, it should have rendered a H1.Again, this behavior is not consistent. So...

bdanielatl_0-1736861864741.png
Administration & Architecture
Reply
  • 229 Views
  • 1 replies
  • 3 kudos
View Replies
Latest Reply
Alberto_Umana
Databricks Employee
  • 3 kudos

Hi @bdanielatl, Thank you for reporting the issue with markdown cells not rendering consistently. This appears to be a known issue that has been encountered by other users as well. I will report it internally.

  • 3 kudos
kpriya
by • New Contributor II
  • 149 Views
  • 1 replies
  • 2 kudos

Missing Permission option for DLT pipline

I do not see permission option in DLT pipeline's page kebab menu as well, can some one help on this?

Administration & Architecture
Reply
  • 149 Views
  • 1 replies
  • 2 kudos
View Replies
Latest Reply
Alberto_Umana
Databricks Employee
  • 2 kudos

Hi @kpriya, Ensure you are on the correct page where the pipeline is listed. Click on the pipeline name to go into its details Once you are on the pipeline details page, look for the kebab menu (three vertical dots) associated with the pipeline. This...

  • 2 kudos
arun_prakash
by • New Contributor
  • 209 Views
  • 1 replies
  • 0 kudos

Databricks Apps not working in postman

I have a question regarding Databricks Apps. I have deployed my databricks Apps, and its working on my laptop, but when I try to open the same url in my mobile its redirecting to databricks signin page, and also its not working through postman as wel...

Administration & Architecture
Reply
  • 209 Views
  • 1 replies
  • 0 kudos
View Replies
Latest Reply
Walter_C
Databricks Employee
  • 0 kudos

Is this issue specifically with the Databricks Apps am I right? Are you getting and erro message?

  • 0 kudos
SravanThotakura
by • New Contributor II
  • 329 Views
  • 3 replies
  • 0 kudos

Resolved! Spark Job fails with No plan for OptimizedForeachBatchFastpath

Hi Team,I am trying to run a job on Databricks cluster 14.3LTS which streams data from parquet to custom sink. I am facing below error. The same code used to work a month back, however i am facing this issue recently.org.apache.spark.SparkException: ...

Administration & Architecture
Reply
  • 329 Views
  • 3 replies
  • 0 kudos
View Replies
Latest Reply
Walter_C
Databricks Employee
  • 0 kudos

Great to hear the issue got fixed.

  • 0 kudos
2 More Replies

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group
Labels
li.common.scroll-to.top