- 2447 Views
- 3 replies
- 0 kudos
Not able to perform update in delta table in databricks using 3 tables
Hi,I am able to perform merge from 2 tables but have requirement to update table based on 3 tables like following query.update a set a.name=b.namefrom table1 a inner join table2 b on a.id=b.idinner join table3 c on a.id=c.idThanks in advance..
- 2447 Views
- 3 replies
- 0 kudos
- 0 kudos
Hi @upendra kumar sharma​ Help us build a vibrant and resourceful community by recognizing and highlighting insightful contributions. Mark the best answers and show your appreciation!Thanks and Regards
- 0 kudos
- 4355 Views
- 4 replies
- 0 kudos
Error in SQL statement: AnalysisException: The query operator `UpdateCommandEdge` contains one or more unsupported expression types Aggregate, Window or Generate.
com.databricks.backend.common.rpc.DatabricksExceptions$SQLExecutionException: org.apache.spark.sql.AnalysisException: The query operator `UpdateCommandEdge` contains one or more unsupportedexpression types Aggregate, Window or Generate.Invalid expres...
- 4355 Views
- 4 replies
- 0 kudos
- 0 kudos
Hi @Pradeep Chauhan​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answe...
- 0 kudos
- 3445 Views
- 4 replies
- 3 kudos
Is the Model tab not available in Databricks Community cloud edition?
I am currently doing the Scalable Machine Learning Course and I observed that the menu options available in the videos are a bit different from what I have on my edition.Is it because I am using the Community edition or there is some setting to make ...
- 3445 Views
- 4 replies
- 3 kudos
- 3 kudos
Hi @Pradeep Gadkari​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answe...
- 3 kudos
- 5895 Views
- 4 replies
- 0 kudos
Slow updates/upserts in Delta tables
When using Delta tables with DBR jobs or even with DLT pipelines, the upserts (especially updates) (on key and timestamp) are taking quite higher than expected time to update the files/tables data (~2 mins for even 1 record poll) (Inserts are lightni...
- 5895 Views
- 4 replies
- 0 kudos
- 0 kudos
Hi @Surya Agarwal​ Hope everything is going great.Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us so...
- 0 kudos
- 3415 Views
- 3 replies
- 6 kudos
Databricks Web Terminal Now you can direct access the web terminal direct from the view menu this is very handy while working on the terminal.Â
Databricks Web TerminalNow you can direct access the web terminal direct from the view menu this is very handy while working on the terminal.
- 3415 Views
- 3 replies
- 6 kudos
- 6 kudos
Hi @Ajay Pandey​ Hope everything is going great.Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us so w...
- 6 kudos
- 26258 Views
- 5 replies
- 0 kudos
How to upload large files to Databricks? and how to unzip files successfully?
I have two JSON files, one ~3 gb and one ~5 gb. I am unable to upload them to databricks community edition as they exceed the max allowed up-loadable file size (~2 gb). If I zip them I am able to upload them, but I am also having issues figuring out ...
- 26258 Views
- 5 replies
- 0 kudos
- 0 kudos
Hi @Sage Olson​ Hope everything is going great.Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us so we...
- 0 kudos
- 2797 Views
- 2 replies
- 0 kudos
DLT live Table-Incremental Refresh
Hi All,In our ETL Framework, we have four layers Raw, Foundation ,Trusted & Unified .In raw we are copying the file in JSON Format from a source, using ADF pipeline .In the next Layer(i.e. Foundation) we are flattening the Json Files and converting t...
- 2797 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi @DataBricks_User9 c​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best an...
- 0 kudos
- 4555 Views
- 5 replies
- 0 kudos
Need urgent help while Trying to update the below statement from spark SQL. Getting error
spark.sql('UPDATE kct SET ABC_basevalue = kct.ABC_KBETR * ABS(T5.UKUSA) FROM cond_azure_try_delta kct INNER JOIN VW_TCU_delta AS T5 ON kct.WRS = T5.TCU AND kct.ABC_konwa=t5.fc INNER JOIN tcu_delta t ON(kct.WRS=t.FC) WHERE kct.marked="1"')
- 4555 Views
- 5 replies
- 0 kudos
- 0 kudos
Hi @sreekanth kesaram​ Hope everything is going great.Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell u...
- 0 kudos
- 3472 Views
- 3 replies
- 2 kudos
Why is pytorch cuda total memory not aligned with the memory size of GPU cluster I created?
No matter GPU cluster of which size I create, cuda total capacity is always ~16 Gb. Does anyone know what is the issue?The code I use to get the total capacity:torch.cuda.get_device_properties(0).total_memory
- 3472 Views
- 3 replies
- 2 kudos
- 2 kudos
Hi @Simon Zhang​ Hope everything is going great.Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us so w...
- 2 kudos
- 17540 Views
- 10 replies
- 5 kudos
How to use/access in a python notebook a scala library installed from JAR file?
I'm using Azure Event Hubs Connector https://github.com/Azure/azure-event-hubs-spark to connect an Even Hub.When I install this library from Maven , then everything works, I can access lib classes using JVM:connection_string = "<connection_string>" s...
- 17540 Views
- 10 replies
- 5 kudos
- 5 kudos
Hi @blackcoffee AR​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answer...
- 5 kudos
- 18594 Views
- 2 replies
- 1 kudos
Get total size of data in a catalog and schema in Unity Catalog
For a KPI dashboard, we need to know the exact size of the data in a catalog and also all schemas inside the catalogs. What is the best way to do this? We tried to iterate over all tables and sum the sizeInBytes using the DESCRIBE DETAIL command for ...
- 18594 Views
- 2 replies
- 1 kudos
- 1 kudos
Hi @Anant Pingle​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers ...
- 1 kudos
- 7956 Views
- 2 replies
- 0 kudos
Databricks Certified Data Engineer Professional
Hello,May i know when can i expect The instructor-led Advanced Data Engineering with Databricks course for Databricks Certified Data Engineer Professional will be available in 2023..or any references to prepare would be great.
- 7956 Views
- 2 replies
- 0 kudos
- 0 kudos
@sarath endluri​ I saw an ILT training through my Partners Academy, scheduled for April 24-27. Today I just got an e-mail saying it was canceled because they needed to update the content. I asked for further info and got told it's going to be a while...
- 0 kudos
- 1714 Views
- 1 replies
- 0 kudos
Maintaining Custom State in Structured Streaming
I am consuming an IoT stream with thousands of different signals using Structured Streaming. During processing of the stream, I need to know the previous timestamp and value for each signal in the micro batch. The signal stream is eventually written ...
- 1714 Views
- 1 replies
- 0 kudos
- 0 kudos
@Suteja Kanuri​ Tried the above on streaming DFBut facing the below errorAttributeError: 'DataFrame' object has no attribute 'groupByKey'Can you please let me know DBR runtime
- 0 kudos
- 1857 Views
- 4 replies
- 0 kudos
I have not received my voucher yet.Even mails are not replied​
I have not received my voucher yet.Even mails are not replied​
- 1857 Views
- 4 replies
- 0 kudos
- 0 kudos
Hi @Rituparna Das​ Hope everything is going great.Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us so...
- 0 kudos
- 5293 Views
- 2 replies
- 0 kudos
How to trigger jobs after previous job are executed (I want to orchestrate jobs not tasks)?
I know scheduling options are there, but it doesn't consider interdependency like don't execute job2 unless job1 has been executed.I know, script based, or API based triggers are also there, but I am looking for UI based triggers like how we can orch...
- 5293 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi @satyam rastogi​ Hope everything is going great.Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us s...
- 0 kudos
| User | Count |
|---|---|
| 1644 | |
| 793 | |
| 553 | |
| 349 | |
| 287 |
