- 1857 Views
- 1 replies
- 0 kudos
Cluster Auto Termination Best Practices
Are there any recommended practices to set cluster auto termination for cost optimization?
- 1857 Views
- 1 replies
- 0 kudos
- 0 kudos
Hello @dataailearner , Greetings of the day! Here are a few steps that you can follow for cost optimizations: 1. Choose the most efficient compute size: Databricks runs one executor per worker node. The total number of cores across all executors is a...
- 0 kudos
- 3866 Views
- 1 replies
- 0 kudos
Databricks Custom model Serving endpoint Failing
Hello all, I have created a custom model serving endpoint in Azure databricks. This endpoint connects with the AzureopenAI model and Azure postgres connection.All of these Azure services are with Private endpoints. When I run this notebook,I am able ...
- 3866 Views
- 1 replies
- 0 kudos
- 758 Views
- 1 replies
- 0 kudos
VSCode Databricks Extension
Hi all,I've been trying to sync my VSCode IDE with our Databricks GCP workspace using the Databricks extension. I am able to connect authenticate my account and workspace and find our clusters. However, when I try to sync a destination it throws a st...
- 758 Views
- 1 replies
- 0 kudos
- 0 kudos
@Retired_mod thanks for you response.I am not running through a proxy. At least, not on purpose. How do I know if I am running through a proxy? And where can I find the values of <proxy_url> and <port> so that I can try restarting my VSCode.I have tr...
- 0 kudos
- 22415 Views
- 0 replies
- 0 kudos
Identity column has null values
I created a table in databricks using a dbt model pre hook CREATE TABLE IF NOT EXISTS accounts ( account_id BIGINT GENERATED ALWAYS AS IDENTITY, description STRING other columns)I use the same dbt model to merge values into this table in the post...
- 22415 Views
- 0 replies
- 0 kudos
- 1235 Views
- 0 replies
- 0 kudos
UCX Installation Error
While downloading and installing ucx from a shell code,I am facing the below error. Can anyone provide a solution[i] Creating isolated Virtualenv with Python: /c/Program Files/Python312/pythonActual environment location may have moved due to redirect...
- 1235 Views
- 0 replies
- 0 kudos
- 710 Views
- 0 replies
- 0 kudos
Fetching CPU and memory data using REST APIs
Hi,I am trying to fetch CPU and memory details from Databricks. Are there any APIs present to which I can connect using postman and fetch these details?
- 710 Views
- 0 replies
- 0 kudos
- 3847 Views
- 3 replies
- 1 kudos
Setting up Unity Catalog in Azure
Trying to create a metastore that will be connected to an external storage (ADLS) but we don't have the option to create a new metastore in 'Catalog' tab in the UI. Based on some research, we see that we'll have to go into "Manage Account" and then c...
- 3847 Views
- 3 replies
- 1 kudos
- 1 kudos
I have been wrestling with this question for days now. I seem to be the only one with this question so I am sure I am doing something wrong. I am trying to create a UC metastore but there is not an option in "Catalog" to create a metastore. This s...
- 1 kudos
- 836 Views
- 0 replies
- 0 kudos
Failed deploying bundle via gitlab - Request failed for POST
I'm encountering an issue in my .gitlab-ci.yml file when attempting to execute databricks bundle deploy -t prod. The error message I receive is: Error: Request failed for POST <path>/state/deploy.lockInterestingly, when I run the same command locally...
- 836 Views
- 0 replies
- 0 kudos
- 15018 Views
- 6 replies
- 0 kudos
Renaming the database Name in Databricks
Team,Initially our team created the databases with the environment name appended. Ex: cust_dev, cust_qa, cust_prod.I am looking to standardize the database name as consistent name across environments. I want to rename to "cust". All of my tables are ...
- 15018 Views
- 6 replies
- 0 kudos
- 0 kudos
You can also use “CASCADE” to drop schema and tables as well. It is recursive.
- 0 kudos
- 635 Views
- 0 replies
- 0 kudos
How to confirm a workspace ID via an api token?
Hello! We are integrating with Databricks and we get the API key, workspace ID, and host from our users in order to connect to Databricks. We need the to validate the workspace ID because we do need it outside of the context of the API key (with webh...
- 635 Views
- 0 replies
- 0 kudos
- 696 Views
- 1 replies
- 0 kudos
Custom python package iin Notebook task using bundle
Hi mates!I'n my company, we are moving our pipelines to Databricks bundles, our pipelines use a notebook that receives some parameters.This notebook uses a custom python package to apply the business logic based on the parameters that receive.The thi...
- 696 Views
- 1 replies
- 0 kudos
- 0 kudos
Solved understanding the databricks.yml configuration!
- 0 kudos
- 905 Views
- 0 replies
- 0 kudos
Spot label in pool even though the configuration selected is all on-demand
Why is there a spot label in pool even though the configuration selected is all on-demand? can someone explain?
- 905 Views
- 0 replies
- 0 kudos
- 405 Views
- 0 replies
- 0 kudos
Is there any specific error you are receiving when running the init script? Does the run complete st
Is there any specific error you are receiving when running the init script? Does the run complete start up or fail due to the init script?
- 405 Views
- 0 replies
- 0 kudos
- 460 Views
- 0 replies
- 0 kudos
Eorror in perform a merge inside a streaming foreachbatch using the command: microBatchDF._jdf.s
I'm trying to perform a merge inside a streaming foreachbatch using the command: microBatchDF._jdf.sparkSession().sql(self.merge_query)Streaming
- 460 Views
- 0 replies
- 0 kudos
- 570 Views
- 0 replies
- 0 kudos
ipywidgets not loading in DataBricks Community Edition
Hi All,I am trying to run commands with ipywidgets but it just says :Loading widget...Same error occurs even when I re run the cell.DataBricks version used : 14.2
- 570 Views
- 0 replies
- 0 kudos
