- 1346 Views
- 1 replies
- 0 kudos
Cannot Get Query Results in SQL Alerts
Example Query:select name, date from errors;Now i want to trigger an alert if count is greater than 1, and a notification should be sent to slack with output rows (name and date values). Even if i use {{QUERY_RESULT_ROWS}}, it only gives value after ...
- 1346 Views
- 1 replies
- 0 kudos
- 0 kudos
Note I have not tried this myself, but can you try the following and let me know if it helps: Create the query, so SELECT name, date FROM errors;Set up the alert. Set the condition to trigger the alert when the count of rows is greater than 1.Create ...
- 0 kudos
- 902 Views
- 1 replies
- 0 kudos
Facing Some Issues with Tablet PC and Databricks Product – Any Advice?
Hello everyone,I’m having some trouble using Databricks SQL Analytics v2.1 on my tablet PC, and I was wondering if anyone here has had similar experiences or could offer some advice.The main issues I’m facing are:Performance Slowdowns: When I run com...
- 902 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @jonathanjone , 1 - Performance Slowdowns could be because of the warehouse size and the query count, if there are many queries, the warehouse has a limit of 10 query in parallel, else you see the query being queued. You could also check if the q...
- 0 kudos
- 1306 Views
- 1 replies
- 1 kudos
Resolved! Has the numUpdateRetryAttempts property been deprecated?
I noticed there is a numUpdateRetryAttempts property mentioned in the the document https://learn.microsoft.com/en-us/azure/databricks/delta-live-tables/properties used for configing the retry times of any DLT pipeline, but I cannot find it in the DL...
- 1306 Views
- 1 replies
- 1 kudos
- 1 kudos
According to the Delta Live Tables properties reference, pipelines.numUpdateRetryAttempts is a recognized configuration parameter. It specifies the maximum number of attempts to retry an update before failing the update when a retryable failure occur...
- 1 kudos
- 2771 Views
- 3 replies
- 0 kudos
Implementing Conditional Logic for Dependent Tasks Using SQL Output and Task Values
Hi team,I'm working on setting up a workflow with task dependencies where a subsequent task should execute conditionally, based on the result of a preceding SQL task. Specifically, I need to evaluate an if/else condition on the output of the SQL quer...
- 2771 Views
- 3 replies
- 0 kudos
- 1131 Views
- 1 replies
- 0 kudos
QGIS python command in Databricks notebook
Hello, I would like to run a qgis python script in a databricks notebook. Currently, databricks doesn't recognize the qgis package. For example, i'd like to run this small example : from qgis.core import *# Supply path to qgis install locationQgsAppl...
- 1131 Views
- 1 replies
- 0 kudos
- 0 kudos
QGIS is not directly usable in Scala or Spark environments as it is a standalone Geographic Information System (GIS) application, installing and using it directly within Databricks may not be straightforward due to the specific environment and depend...
- 0 kudos
- 1617 Views
- 4 replies
- 0 kudos
MetaStore Issues
Hi,I am using Student Account in Azure and created databricks workspace. I am trying to locate Manage Account to create MetaStore but I am not successful. Would need your help.no Manage Account option available.If you see I am an Admin, Thanks for a...
- 1617 Views
- 4 replies
- 0 kudos
- 0 kudos
Databricks student account, may have certain limitations compared to a full Azure Databricks account. For student accounts, you might not have the necessary permissions to create a Unity Catalog metastore. Typically, creating and managing a metastore...
- 0 kudos
- 8379 Views
- 5 replies
- 1 kudos
Cannot create storage credential without Contributor role
Hello,I am trying to create a Storage Credential. I have created the access connector and gave the managed identity "Storage Blob Data Owner" permissions. However when I want to create a storage credential I get the following error:Creating a storage...
- 8379 Views
- 5 replies
- 1 kudos
- 1 kudos
Did someone find a solution?{ "error_code": "RESOURCE_DOES_NOT_EXIST", "message": "Refresh token not found for userId: Some(2302042022180399)", "details": [ { "@type": "type.googleapis.com/google.rpc.RequestInfo", "request_id": "d731471b-b...
- 1 kudos
- 1343 Views
- 1 replies
- 0 kudos
Scala notebooks don't automatically print variables
It seems like with Scala notebook if I declare some variables or have important statements then the cell runs it will automatically print out the variables and import statements. Is there a way to disable this, so only explicit println are output?
- 1343 Views
- 1 replies
- 0 kudos
- 0 kudos
There isn't a configuration that can be set to True/False and control this behavior for some statements. This output is part of Databrick's interactive notebook design, where all evaluated statements—such as imports, variable declarations, and expres...
- 0 kudos
- 7790 Views
- 7 replies
- 4 kudos
Resolved! Databricks SQL: catalog of each query
Currently, we are migrating from hive metastore to UC. We have several dashboards and a huge number of queries whose catalogs have been set to hive_metastore and using <db>.<table> access pattern.I'm just wondering if there's a way to switch catalogs...
- 7790 Views
- 7 replies
- 4 kudos
- 4 kudos
May you can also have a look here, if you need hot fix https://github.com/databrickslabs/ucx
- 4 kudos
- 1747 Views
- 2 replies
- 0 kudos
Glob pattern for copy into
I am trying to load some files in my Azure storage container using copy into method. The files have a naming convention of "2023-<month>-<date> <timestamp>".csv.gz. All the files are in one folder. I want to load only files for month 2. So I've used...
- 1747 Views
- 2 replies
- 0 kudos
- 0 kudos
TL;DR Try removing the trailing slash in the FROM value. The trailing slash in FROM confuses the URI parser, making it think that PATTERN might be an absolute path rather than a relative one. The error message points to a problem not with respect to ...
- 0 kudos
- 30450 Views
- 12 replies
- 2 kudos
Resolved! Dbfs init script migration
I need help with migrating from dbfs on databricks to workspace. I am new to databricks and am struggling with what is on the links provided.My workspace.yml also has dbfs hard-coded. Included is a full deployment with great expectations.This was don...
- 30450 Views
- 12 replies
- 2 kudos
- 5797 Views
- 5 replies
- 0 kudos
Default maximum spark streaming chunk size in delta files in each batch?
working with delta files spark structure streaming , what is the maximum default chunk size in each batch?How do identify this type of spark configuration in databricks?#[Databricks SQL]​ #[Spark streaming]​ #[Spark structured streaming]​ #Spark​
- 5797 Views
- 5 replies
- 0 kudos
- 0 kudos
doc - https://docs.databricks.com/en/structured-streaming/delta-lake.html Also, what is the challenge while using foreachbatch?
- 0 kudos
- 1616 Views
- 4 replies
- 1 kudos
workflow fails when ran using a job cluster but not in shared cluster
Hi,We have a workflow which calls 3 notebooks and when we run this workflow using shared cluster it runs fine, but when ran with job cluster , one of the notebooks fail.This notebook uses sql function Lpad and looks like it errors because of it. Has ...
- 1616 Views
- 4 replies
- 1 kudos
- 3539 Views
- 5 replies
- 0 kudos
Delta live type 2 scd Liquid clustering on Start and end dates
I've created a DLT pipeline that creates type 2 SCDs and often the __Start_at and __end_at columns are beyond the first 32 columns for stat collection.I'd like to add these columns to liquid clustering without increasing the number of columns in the ...
- 3539 Views
- 5 replies
- 0 kudos
- 0 kudos
Are these responses generated by chatgpt? They don't answer the question and very much have the tone of generative AI
- 0 kudos
- 1285 Views
- 2 replies
- 0 kudos
Error connecting to MySQL from Databricks: (2003, "Can't connect to MySQL server")
Hello Community,I'm facing an issue connecting to a MySQL database hosted on AWS RDS from within a Data bricks notebook. My Python script to connect to MySQL works fine locally, but when I run it in Data bricks, I receive this error:Error connecting ...
- 1285 Views
- 2 replies
- 0 kudos
| User | Count |
|---|---|
| 1644 | |
| 793 | |
| 553 | |
| 349 | |
| 287 |
