cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Himanshu_90
by New Contributor III
  • 6879 Views
  • 8 replies
  • 7 kudos

Databricks sql not able to evaluate expression current_user

Hi,I have a table as below:create table default.test_user(ID bigint NOT NULL GENERATED BY DEFAULT AS IDENTITY (START WITH 1 INCREMENT BY 1),usr1 varchar(255) NOT NULL,ts1 timestamp NOT NULL,usr2 varchar(255) NOT NULL,ts2 timestamp NOT NULL) USING Del...

  • 6879 Views
  • 8 replies
  • 7 kudos
Latest Reply
Anonymous
Not applicable
  • 7 kudos

Hi @Himanshu Agrawal​ Hope everything is going great.Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us...

  • 7 kudos
7 More Replies
Sandy84
by New Contributor II
  • 5734 Views
  • 3 replies
  • 2 kudos

Need help skipping previously executed cells in a failed Databricks job calling a notebook with multiple SQL cells

In Azure databricks, I have a job that calls a notebook which has multiple cells with sql queries. In case of any cell fails and when we restart the databricks job then how to skip previous cell which already ran and start only from the failed cell? ...

  • 5734 Views
  • 3 replies
  • 2 kudos
Latest Reply
Anonymous
Not applicable
  • 2 kudos

Hi @Sandip Rath​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers y...

  • 2 kudos
2 More Replies
Rami2023
by New Contributor II
  • 10601 Views
  • 1 replies
  • 0 kudos

Read CSV file using SQL

Hi, I am trying to reverse engineer to get to the source file for a table. Looking at the query history, I came across SQL string which loads data from file to table, however the code looks little mystery to me. I haven't come across idbfs, Can someb...

  • 10601 Views
  • 1 replies
  • 0 kudos
Anonymous
by Not applicable
  • 748 Views
  • 1 replies
  • 2 kudos

www.databricks.com

Dear Community - @Youssef Mrini​ will answer all your questions on April 19, 2023 from 9:00am to 10:00am GMT during the Databricks EMEA Office Hours.Make sure to join this amazing 'Ask Me Anything' session by Databricks - https://www.databricks.com/r...

  • 748 Views
  • 1 replies
  • 2 kudos
Latest Reply
youssefmrini
Databricks Employee
  • 2 kudos

It was a successful office hours. Make sure to join the next one.

  • 2 kudos
AEW
by New Contributor II
  • 18874 Views
  • 8 replies
  • 2 kudos

Resolved! Help with Parameters in Databricks SQL

I am following the documentation on query parameters (https://docs.databricks.com/sql/user/queries/query-parameters.html) and it's not working for me. I can't get a parameter inserted at the text caret and I can't get cmd + p to work. I've changed sp...

  • 18874 Views
  • 8 replies
  • 2 kudos
Latest Reply
labromb
Contributor
  • 2 kudos

I don't see the need for spaces around the parameter name... I have just been typing {{param_name}} and it appears automatically in SQL editor

  • 2 kudos
7 More Replies
Anonymous
by Not applicable
  • 2453 Views
  • 2 replies
  • 2 kudos

Hello Everyone, I'm interested to learn about the certifications you're pursuing to enhance your skills. Sharing your goals can inspire those ...

Hello Everyone,I'm interested to learn about the certifications you're pursuing to enhance your skills. Sharing your goals can inspire those who may have started their certification journey but struggled with motivation. Personally, I recently comple...

  • 2453 Views
  • 2 replies
  • 2 kudos
Latest Reply
FJ
Contributor III
  • 2 kudos

I'm trying the Data Engineering professional exam at the end of the month. It's like a shot in the dark because no practice exams stop are available and from what I've seen online from people who already passed it, the Advanced Data Engineering with ...

  • 2 kudos
1 More Replies
Julie1
by New Contributor II
  • 8022 Views
  • 2 replies
  • 1 kudos

Resolved! Query data not showing in custom alert notifications and QUERY_RESULT_ROWS

I've set up a custom alert notification for one of my Databricks SQL queries, and it triggers correctly, but I'm not able to get the actual results of the query to appear in the notification email. I've followed the example/template in the custom ale...

  • 8022 Views
  • 2 replies
  • 1 kudos
Latest Reply
-werners-
Esteemed Contributor III
  • 1 kudos

The actual query results are not displayed in the alert unfortunately. You can pass the alert condition etc, but not the raw results of the underlying query.I hope this will be added in the future.A workaround is to add a link to the query, so the r...

  • 1 kudos
1 More Replies
labromb
by Contributor
  • 3343 Views
  • 2 replies
  • 0 kudos

Getting Databricks SQL dashboard to recognise change to an underlying query

Hi CommunityScenario:I have created a query in Databricks SQL, built a number of visualisations from it and published them to a dashboard. I then realise that I need to add another field to the underlying query that I want to then leverage as a dashb...

  • 3343 Views
  • 2 replies
  • 0 kudos
Latest Reply
youssefmrini
Databricks Employee
  • 0 kudos

Can you take a screenshot ?

  • 0 kudos
1 More Replies
codeexplorer
by New Contributor II
  • 9504 Views
  • 4 replies
  • 0 kudos

Update record in databricks sql table from C#.Net in visual studio 2022 using ODBC

I am trying to make a backend method call work which connects to database and updates the record in a table. The method call works perfectly and it is not throwing any error but at the same time it does not update any record in the table either.Note:...

  • 9504 Views
  • 4 replies
  • 0 kudos
Latest Reply
codeexplorer
New Contributor II
  • 0 kudos

I found a temporary work around. Instead of passing the value through the parameters, I passed the value directly in query like below. I know it is not the ideal way but at this time this is working. If I do not pass the value as shown below, the log...

  • 0 kudos
3 More Replies
Vijesh
by New Contributor II
  • 4310 Views
  • 5 replies
  • 1 kudos

parsing error in Databricks SQL endpoint

I have two tables EMPLOYEE & EMPLOYEE_ROLE. I'm trying to Update a column with a value from another column. I'm using SQL server join but i get an error - [parse_syntax_error] Syntax error at or near 'FROM' line 3. UPDATE CSET C.title = B.title FROM ...

  • 4310 Views
  • 5 replies
  • 1 kudos
Latest Reply
sensanjoy
Contributor
  • 1 kudos

Hi @Vijesh V​ Try to use merge into to perform cdc between tables :MERGE INTO target aUSING source bON {merge_condition}WHEN MATCHED THEN {matched_action}WHEN NOT MATCHED THEN {not_matched_action}

  • 1 kudos
4 More Replies
Gajji
by New Contributor
  • 2965 Views
  • 2 replies
  • 0 kudos

How to link 2 Dashboards

Hi, I have 2 dashboards. 1st Dashboard has a table with two columns, 'Country' and 'NumberOfMajorCities'. When I click on any row of this table/click on 'NumberOfMajorCities' field of a row, I would like to jump to 2nd Dashboard, where I display all ...

  • 2965 Views
  • 2 replies
  • 0 kudos
Latest Reply
Anonymous
Not applicable
  • 0 kudos

Hi @Ghazanfar Sl​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers ...

  • 0 kudos
1 More Replies
Hubert-Dudek
by Esteemed Contributor III
  • 2120 Views
  • 1 replies
  • 10 kudos

databricks SQL now supports getting the SQLSTATE code of a query to identify errors. You can use the e.getSqlState() method in a try/catch block to ge...

databricks SQL now supports getting the SQLSTATE code of a query to identify errors. You can use the e.getSqlState() method in a try/catch block to get the five-character code that indicates the success or failure of an SQL command. I am still thinki...

Untitled
  • 2120 Views
  • 1 replies
  • 10 kudos
Latest Reply
jose_gonzalez
Databricks Employee
  • 10 kudos

Thank you for sharing @Hubert Dudek​ 

  • 10 kudos
Anonymous
by Not applicable
  • 908 Views
  • 1 replies
  • 2 kudos

Join our Community Social Group and Never Miss a Beat! Are you looking to connect with like-minded individuals and stay on top of the latest news and ...

Join our Community Social Group and Never Miss a Beat!Are you looking to connect with like-minded individuals and stay on top of the latest news and events in your community? Look no further than our special group on Community called the "Community S...

  • 908 Views
  • 1 replies
  • 2 kudos
Latest Reply
Ajay-Pandey
Esteemed Contributor III
  • 2 kudos

@Rishabh Pandey​ 

  • 2 kudos
Anonymous
by Not applicable
  • 7771 Views
  • 4 replies
  • 4 kudos

How to create a new group in the Databricks community? Dear esteemed community users, It is with great pleasure that we inform you of an important upd...

How to create a new group in the Databricks community?Dear esteemed community users,It is with great pleasure that we inform you of an important update regarding the creation of Groups on Community. As part of our continuous efforts to enhance your e...

  • 7771 Views
  • 4 replies
  • 4 kudos
Latest Reply
Anonymous
Not applicable
  • 4 kudos

HI @Hubert Dudek​  and @Ratna Chaitanya Raju Bandaru​ : Thanks for pointing this out. This is going to be a design decision which we will take after looking into the ask carefully. Thanks for getting the conversation going. This really helps us.

  • 4 kudos
3 More Replies
oleole
by Contributor
  • 9718 Views
  • 3 replies
  • 2 kudos

Resolved! Using "FOR XML PATH" in Spark SQL in sql syntax

I'm using spark version 3.2.1 on databricks (DBR 10.4 LTS), and I'm trying to convert sql server sql query to a new sql query that runs on a spark cluster using spark sql in sql syntax. However, spark sql does not seem to support XML PATH as a functi...

input output
  • 9718 Views
  • 3 replies
  • 2 kudos
Latest Reply
oleole
Contributor
  • 2 kudos

Posting the solution that I ended up using:%sql DROP TABLE if exists UserCountry; CREATE TABLE if not exists UserCountry ( UserID INT, Country VARCHAR(5000) ); INSERT INTO UserCountry SELECT L.UserID AS UserID, CONCAT_WS(',', co...

  • 2 kudos
2 More Replies
Labels