- 487 Views
- 1 replies
- 2 kudos
Resolved! Need help in creating compute cluster
Hi everyone,I hope you’re doing well.I’m currently facing an issue while trying to create a cluster in Azure Databricks (Personal). Despite trying various approaches, I keep encountering the following error message:The VM size you are specifying is n...
- 487 Views
- 1 replies
- 2 kudos
- 2 kudos
Hi @Naveen_Sequeira ,If you increased quota and it didn't help then just try to use different region. In Azure, some machines are sometimes unavailable due to regional capacity issue. They should be available in another region though.You can try with...
- 2 kudos
- 287 Views
- 1 replies
- 0 kudos
Passing a list of images on AI_Query
HiI would like to know how can I pass a list of images to ai_query on the same call, It will be helpful to extract some metadata information on a multipage document, where I want to know if different images to see if it belongs to the same document.T...
- 287 Views
- 1 replies
- 0 kudos
- 0 kudos
Hello @Andreyai Has your issue got resolved?
- 0 kudos
- 3563 Views
- 11 replies
- 2 kudos
Issue with Non-Bulk Inserts Using JDBC on Databricks Runtime 14.1
Hello team,I am experiencing an issue with insert operations on Databricks using the JDBC driver. In my SAS Viya system, the DatabricksJDBC42.jar driver version 2.6.40 is configured. I’ve noticed that, up to Databricks Runtime version 13.1, insert op...
- 3563 Views
- 11 replies
- 2 kudos
- 2 kudos
The PR looks closed for now, any news on this? Also, I was wondering if there's a code snippet available on using this functionality
- 2 kudos
- 770 Views
- 3 replies
- 3 kudos
Resolved! Help to get the swags for the event.
Hello Team,I am hosting my first event on 13th September. The date is final, but the venue is something that I am working on, due to I am not creating the event. I'll be able to create it within a day or two. Can you please help me understand how I ...
- 770 Views
- 3 replies
- 3 kudos
- 3 kudos
Thank you for tagging @BS_THE_ANALYST !Hi @Devanshu ,Thanks for reaching out. I’ve sent you a DM with the swag request details, please share your response so we can move forward and support you better. ThanksRishabh
- 3 kudos
- 4862 Views
- 1 replies
- 0 kudos
the inference table table doesn't get updated
I setup a model serving endpoint and created a monitoring dashboard to monitor its performance. The problem is my inference table doesn't get updated by model serving endpoints. To test the endpoint I use the following codeimport random import time ...
- 4862 Views
- 1 replies
- 0 kudos
- 0 kudos
If the inference table is renamed, schema changed, deleted, or if the creator loses data access/the permissions are missing or have recently changed (for the endpoint creator), inference tables may silently stop updating.
- 0 kudos
- 542 Views
- 1 replies
- 0 kudos
How to install python libraries in DLT using databricks asset bundles
Hi Team,Is there any way we can install python packages to DLT using Databricks asset bundles. resources:pipelines:xysz:name: xyxconfiguration:input_file: test.jsonenv: ${var}permissions:- group_name: ${var}level: CAN_MANAGE- group_name: ${var}level...
- 542 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @nkrom456 ,Add a %pip install command at the top of your DLT pipeline notebook to install the required packages. This will ensure the packages are available on all nodes during pipeline execution.
- 0 kudos
- 384 Views
- 1 replies
- 1 kudos
Performance issue with the Databricks Add-on for Splunk 1.4.2
We are currently using the add-on for Databricks in our on-prem Splunk Enterprise environment. The connection goes directly to the cloud without a proxy. Unfortunately, the add-on is very slow and we lose about 20 seconds with every query. We see th...
- 384 Views
- 1 replies
- 1 kudos
- 1 kudos
It’s likely not Databricks but the Splunk add-on causing the delay. The databricksquery command in version 1.4.2 has known performance issues — most of the lag is inside Splunk while parsing results. Try upgrading to the latest add-on (1.5.x+), monit...
- 1 kudos
- 867 Views
- 2 replies
- 0 kudos
DLT Pipeline Design
I am new to DLT and trying to understand the process. My bronze table will receive incremental data from SAP in real time. In my bronze table, we are not maintaining any history and any data older than 2 weeks will be deleted. This data from bronze w...
- 867 Views
- 2 replies
- 0 kudos
- 0 kudos
The scenario you mentioned can be efficiently managed using a Delta Live Table (DLT) combined with a CDC (Change Data Capture) flow. From what I understand, the bronze table is always in append mode. Check out the link below for more details. Hope th...
- 0 kudos
- 657 Views
- 6 replies
- 2 kudos
Execute Immediate not working to fetch table name based on year
I am trying to pass the year as argument so it can be used in the table name. Ex: there are tables like Claims_total_2021 , Claims_total_2022 and so on till 2025. Now I want to pass the year in parameter , say 2024 and it must fetch the table Claims...
- 657 Views
- 6 replies
- 2 kudos
- 2 kudos
Hi, I believe the solution shared by Martison would fix this issue. In Databricks SQL, when using EXECUTE IMMEDIATE, the SQL string must be a single variable or single string literal, not an inline expression using string concatenation ('...' || clai...
- 2 kudos
- 763 Views
- 6 replies
- 20 kudos
Resolved! Databricks Community Innovators - Program
Hi, I'd like to know what's happened with the Databricks Community innovators program? https://community.databricks.com/t5/databricks-community-innovators/bg-p/databricks-community-news-members Is this still alive? I've applied and emailed: databrick...
- 763 Views
- 6 replies
- 20 kudos
- 20 kudos
Hi Mandy! Thanks for the introduction and update. I'm really looking forward to being a part of everything moving forward. Just echoing what @TheOC mentioned above, I'm happy to help/assist in whatever way I can.Absolutely buzzing based off the info ...
- 20 kudos
- 1253 Views
- 11 replies
- 43 kudos
Databricks Monthly Spotlight - Discontinued?
Hi, I'm curious what's happened with the Databricks monthly spotlight? https://community.databricks.com/t5/databricks-community-innovators/bg-p/databricks-community-news-members I can there hasn't been anyone in the spotlight since April 2025. Has th...
- 1253 Views
- 11 replies
- 43 kudos
- 43 kudos
Hi all,I just wanted to jump in this thread to say how motivating this kind of conversation is. It's great that we've got passion from users and Databricks employees alike to ensure integrity in the Community - it's really reassuring as someone start...
- 43 kudos
- 381 Views
- 3 replies
- 3 kudos
Python
Guys am not able to access pyspark in free edition somebody help me.
- 381 Views
- 3 replies
- 3 kudos
- 3 kudos
@szymon_dybczak it may be that @SinchBhat has the notebook set to SQL as the default language ?Could also, by mistake, be using the SQL editor.@SinchBhat could you provide some screenshots of what your UI looks like please. It'll help to resolve the ...
- 3 kudos
- 325 Views
- 3 replies
- 2 kudos
Resolved! Not able to login Databricks
I am trying to login in databricks but when I enter the OTP it says something went wrong
- 325 Views
- 3 replies
- 2 kudos
- 2 kudos
HI @joypillai ,You want to login to databricks free edition? Could you provide screenshot?
- 2 kudos
- 335 Views
- 1 replies
- 1 kudos
Best practices for reducing noise in data quality monitoring?
Hi all,We’ve been improving our data quality monitoring for several pipelines, but we keep running into the same problem — too many alerts, most of which aren’t actionable. Over time, it becomes harder to trust them.Right now, we’re doing:Freshness c...
- 335 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi @Sifflet, This is genuinely complex—and while you mentioned alerting and monitoring, in my experience the biggest lever to reduce noise is to treat problems at the source (i.e., in the transformation layer). Make the transformations enforce the co...
- 1 kudos
- 759 Views
- 1 replies
- 0 kudos
DLT Pipeline
I am working on DLT pipeline and have one question. As explained on this page (https://docs.databricks.com/aws/en/dlt/tutorial-pipelines?language=Python), we will end up creating 3 tables i.e. customer_cdc_bronze, customer_cdc_clean and customer. Thi...
- 759 Views
- 1 replies
- 0 kudos
- 0 kudos
With this optimized approach, I would suggest creating view for Clean table:1. Bronze Table: Raw CDC data (full storage)2. Clean View: No physical storage - computed on-demand3. Silver Table: Final processed data with SCD2 historyResult: ~67% storage...
- 0 kudos
