- 309 Views
- 2 replies
- 4 kudos
Resolved! How to invoke Databricks AI Assistant from a notebook cell?
Hello Community,I am exploring the Databricks AI Assistant and wondering if there is a way to invoke or interact with it directly from a notebook cell instead of using the workspace sidebar UI.Is there any built-in command (like %assistant) to open o...
- 309 Views
- 2 replies
- 4 kudos
- 4 kudos
@shashankB There are no command like %assistant exists today to interact with Databricks Assistant. As @szymon_dybczak mentioned in the reply the exiting modes that you can interact with Assistant today.Also there is no published Assistant‑specific ...
- 4 kudos
- 2033 Views
- 1 replies
- 1 kudos
Resolved! How to Efficiently Sync MLflow Traces and Asynchronous User Feedback with a Delta Table
I’m building a custom UI table (using Next.js and FastAPI) to display MLflow trace data from a Retrieval-Augmented Generation (RAG) application running on Databricks Managed MLflow 3.0. The table needs to show answer generation speed (from CHAT_MODEL...
- 2033 Views
- 1 replies
- 1 kudos
- 1 kudos
Hello! Here are the answers to your questions: - Yes! See databricks managed mlflow tracing - enable production monitor or endpoint config to collect traces in a delta table - We have example code for implementing async feedback collection - Definit...
- 1 kudos
- 6400 Views
- 3 replies
- 2 kudos
SQL While do loops
Hello. Could you please suggest a workaround for a while do loop in Databricks SQL?WHILE LSTART>0 DO SET LSTRING=CONCAT(LSTRING, VSTRING2)Thank you in advance
- 6400 Views
- 3 replies
- 2 kudos
- 2 kudos
@elgeo Here are two alternatives.1. Use a recursive CTEWITH RECURSIVE loop_cte (lstart, lstring) AS ( SELECT 5 AS lstart, '' AS lstring UNION ALL SELECT lstart - 1, CONCAT(lstring, 'VSTRING2') FROM loop_cte WHERE lstart > 1 ) SELECT * FROM ...
- 2 kudos
- 464 Views
- 1 replies
- 1 kudos
SQL Stored Procedure in Databricks
Hello, Is there a sql server equivalent stored procedure in Databricks which supports while loop along with delay as below. or are there any other alternative to achieve the same.while (select count(*) from schema.mart_daily with (nolock)) = 0 begin ...
- 464 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi @ashishasr ,Yes, support for stored procedures is in public preview:CREATE PROCEDURE | Databricks on AWSIn the definition of stored procedure you can use compound statement ( SQL compound statement (BEGIN ... END) with the definition of the SQL Pr...
- 1 kudos
- 631 Views
- 9 replies
- 6 kudos
Streaming- Results not getting updated on arrival of new files
Hi!I'm trying to stream some files using read_files.format("cloudFiles"). However, when new files arrive, the subsequent SQL query and monitoring graphs are not getting updated. Please suggest.
- 631 Views
- 9 replies
- 6 kudos
- 6 kudos
Hi @AanchalSoni If you are using .readStream, make sure you have set a trigger interval (e.g., .trigger(processingTime='1 minute'))
- 6 kudos
- 2678 Views
- 3 replies
- 4 kudos
Resolved! Best Practices for Managing ACLs on Jobs and Job Clusters in Databricks
Hi all,I’m setting up access control for Databricks jobs and have two questions:Ephemeral Job Clusters: Since job clusters are created at runtime, is it best practice to set ACLs on the job itself? The /api/2.0/permissions/clusters/{cluster_id} endp...
- 2678 Views
- 3 replies
- 4 kudos
- 545 Views
- 9 replies
- 0 kudos
Proposal: Switch to Zstd Compression for Parquet to Reduce S3 Costs
We are thinking to change the Spark configuration for Parquet files to use zstd compression.Configuration: spark.sql.parquet.compression.codec = zstdThis will only affect new data written by our Spark jobs. All existing data will remain compressed wi...
- 545 Views
- 9 replies
- 0 kudos
- 0 kudos
Yes my believe is it should support 7.3 LTS as well, we will prove it with thorough testingThanks for the discussion. Cheers
- 0 kudos
- 535 Views
- 3 replies
- 3 kudos
Resolved! from pyspark.ml.stat import KolmogorovSmirnovTest is not working on Serverless compute.
Hi everyone,I am trying to run a Kolmogorov–Smirnov (KS) test on a Spark DataFrame column in Databricks using the built-in pyspark.ml.stat.KolmogorovSmirnovTest. The goal is to apply the KS test directly on Spark DataFrame data without converting it...
- 535 Views
- 3 replies
- 3 kudos
- 3 kudos
Hi @parthesh24 ,It looks more like KolmogorovSmirnovTest module under the hood is trying to access SparkContext which is not supported in serverless. You can check it yourself by trying to use sparkContext in serverless
- 3 kudos
- 159 Views
- 1 replies
- 1 kudos
Dashboard tagging
How can i tag dashbaord? i do not see any place to add tagging to it?
- 159 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi @pinikrisher ,Unforutnately, you can't. Tagging is currently supported on catalogs, schemas, tables, table columns, volumes, views, registered models, and model version
- 1 kudos
- 687 Views
- 6 replies
- 2 kudos
databricks dashboard deployment error
Hi, i am trying to deploy a dashboard using a bundle and github action. but i am getting an error on CI even though the dashboard is deployed. im using the latest version of CLI from https://raw.githubusercontent.com/databricks/setup-cli/main/install...
- 687 Views
- 6 replies
- 2 kudos
- 2 kudos
@szymon_dybczak ,@nayan_wylde , I checked the permissions and the SP has 'can manage' access on the folder.
- 2 kudos
- 276 Views
- 2 replies
- 2 kudos
Resolved! Lakehouse monitoring dashboard shows no data
Hello, I am replicating the demo for Lakehouse monitoring found here: https://notebooks.databricks.com/demos/lakehouse-monitoring/index.htmlFor some reason, my dashboards show empty, i.e., they say 'no data' - like nothing fits the criteria from the ...
- 276 Views
- 2 replies
- 2 kudos
- 2 kudos
Hello @heli123 Can you share the image again? looks like didnt upload well.
- 2 kudos
- 372 Views
- 3 replies
- 3 kudos
Databricks model serving endpoint returns 403 Unauthorized access to workspace when using service
I deployed a simple Iris model in Databricks Model Serving and exposed it as an endpoint. I’m trying to query the endpoint using a service principal. I can successfully fetch the access token with the following databricks_token() function:def databri...
- 372 Views
- 3 replies
- 3 kudos
- 3 kudos
Hi @szymon_dybczak, Thanks for your comment.One of the admins in this workspace tried using the token generated via client id and secret, and were able to successfully get a response from the serving endpoint using this same above mentioned code.Coul...
- 3 kudos
- 456 Views
- 2 replies
- 0 kudos
How to switch serverless dlt pipeline to cost optimized mode from performance optimized
We have a few serverless dlt pipelines that we want to optimize for cost as we are ok with an increased latency. Where can I change the pipeline to run on cost optimized mode. I dont see this option in UI or API
- 456 Views
- 2 replies
- 0 kudos
- 0 kudos
Hello!To enable cost-optimized mode for your serverless DLT pipeline, switch it to Triggered mode and edit the schedule trigger—there you’ll find the option to disable “Performance optimized.” This setting isn’t visible in the main UI or API unless t...
- 0 kudos
- 39610 Views
- 13 replies
- 6 kudos
SQL Stored Procedure in Databricks
Hello. Is there an equivalent of SQL stored procedure in Databricks? Please note that I need a procedure that allows DML statements and not only Select statement as a function provides.Thank you in advance
- 39610 Views
- 13 replies
- 6 kudos
- 6 kudos
how to connect the stored procedures to power bi report builder, seems like it is not working
- 6 kudos
- 190 Views
- 1 replies
- 2 kudos
Resolved! Access to Databricks partner academy
Hi Team,My company is a Databricks Partner. But I can't get registered for Databricks Partner Academy.I have followed the following steps for Partner Academy RegistrationOpen https://partner-academy.databricks.com/learn in your web browser.Click Logi...
- 190 Views
- 1 replies
- 2 kudos
- 2 kudos
Hi @MauGomes ,Don't worry, you already did the best thing you could. Check below thread with exact same issue . The user submitted a ticket and it was resolved by service desk. So, just wait patiently for reply Solved: authorized to access https://pa...
- 2 kudos
| User | Count |
|---|---|
| 1619 | |
| 790 | |
| 480 | |
| 349 | |
| 287 |
