- 7953 Views
- 11 replies
- 2 kudos
Exam issues
How to approach the databricks team if we facing any problem ​Some time question is not appearing properly so any one know the solution kindly tell me ​
- 7953 Views
- 11 replies
- 2 kudos
- 2 kudos
Hi @S Meghala​ Hope everything is going great.Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us so we ...
- 2 kudos
- 1458 Views
- 2 replies
- 0 kudos
Delta write stream to different folders dynamically based on input file
I have root folder and files are getting ingested in sub folders​ . Want to build a workflow which will write stream based on file being ingested
- 1458 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi @Krishnamoorthy Natarajan​ Thank you for your question! To assist you better, please take a moment to review the answer and let me know if it best fits your needs.Please help us select the best solution by clicking on "Select As Best" if it does.Y...
- 0 kudos
- 4585 Views
- 5 replies
- 0 kudos
The Academy training "Introduction to Delta Live Tables" data source error
Hi,I flow the databricks academy training for "Introduction to Delta Live Tables", but it is cannot setup the environment, when excute first notebook "1_DLT UI Walkthrough", below is the error, who can provide the new data source path for this traini...
- 4585 Views
- 5 replies
- 0 kudos
- 0 kudos
Hi @fan tian​ Hope everything is going great.Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us so we c...
- 0 kudos
- 2070 Views
- 3 replies
- 0 kudos
Databricks-PowerBI-Architecture-Help
Hello Community,We are currently working designing Power BI reports, the data source is databricks. We have all our reporting data in bronze/silver layer of databricks. we want to create summarized/aggregated tables in Gold layer and we want to conne...
- 2070 Views
- 3 replies
- 0 kudos
- 0 kudos
Hi @bodempudi venkat​ Hope everything is going great.Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us...
- 0 kudos
- 2108 Views
- 2 replies
- 0 kudos
What is a good way to implement unit tests using github actions for databricks?
I am trying to use a git template for unit tests on a databricks project. The framework uses pylint, pytest and black to check the code. But I am having a lot of trouble getting the github actions vm to run the code without issues. I have had issues ...
- 2108 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi @Stian Arntsen​ Hope everything is going great.Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us so...
- 0 kudos
- 3616 Views
- 2 replies
- 0 kudos
Creating an external table reference vs creating a view
In a practical sense, what is the difference between creating an external table;create table my_catalog.my_schema.my_favourite_table location 'abfss://path/to/my/dataversus creating a view that references the same dataset;create view my_catalog.my_sc...
- 3616 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi @Nicholas Mead​ Thank you for your question! To assist you better, please take a moment to review the answer and let me know if it best fits your needs.Please help us select the best solution by clicking on "Select As Best" if it does.Your feedbac...
- 0 kudos
- 1718 Views
- 3 replies
- 0 kudos
Performance b/w Managed Table and Un-Managed table
I am using Databricks in Azure. I want to mount ADLS Gen2 on Databricks and create unmanged (external) tables on the mount point. But before that I want to know which will give best performance, is it Managed table (stores data in DBFS root)or Un-ma...
- 1718 Views
- 3 replies
- 0 kudos
- 0 kudos
Hi @Paramesh Malla​ Thank you for your question! To assist you better, please take a moment to review the answer and let me know if it best fits your needs.Please help us select the best solution by clicking on "Select As Best" if it does.Your feedba...
- 0 kudos
- 1047 Views
- 2 replies
- 0 kudos
github to adb workspace linkage
i have a workspace in my adb and in that workspace i have folder contains lots of notebook , for backup purpoe i want to copy all the notebook in my github repo, how can i do that in one shot ?
- 1047 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi @suman mukherjee​ Thank you for your question! To assist you better, please take a moment to review the answer and let me know if it best fits your needs.Please help us select the best solution by clicking on "Select As Best" if it does.Your feedb...
- 0 kudos
- 6480 Views
- 2 replies
- 3 kudos
Getting error "User is not an owner of Account" when creating a storage credential in Azure Databricks
I'm using Azure Databricks.I've followed this guide to create an Azure Storage Account and an Access Connector for Azure Databricks. I've given the `Storage Blob Data Contributor` role to the Access Connector in the Storage Account. When I go to the ...
- 6480 Views
- 2 replies
- 3 kudos
- 3 kudos
Hi @Alex Loddengaard​ Hope everything is going great.Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us...
- 3 kudos
- 2146 Views
- 2 replies
- 0 kudos
Delta Live Table Merge/Upserts
Hello,I am trying to create a basic DLT pipeline which does an incremental load. First time it runs perfectly without any issues. However when there are records to be updated, the pipeline fails with the following error:"Flow silver has FAILED fatall...
- 2146 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi @Ram LH​ Thank you for your question! To assist you better, please take a moment to review the answer and let me know if it best fits your needs.Please help us select the best solution by clicking on "Select As Best" if it does.Your feedback will ...
- 0 kudos
- 6911 Views
- 14 replies
- 23 kudos
How to secure all clusters and then start running the code
When there are slow nodes, sometimes a job needs to resize its number of clusters to reach the required number of nodes. Is there any way to make sure no code is running before all nodes are secured? Thank you!
- 6911 Views
- 14 replies
- 23 kudos
- 23 kudos
Hi @Leo Bao​ Hope everything is going great.Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us so we ca...
- 23 kudos
- 1606 Views
- 3 replies
- 0 kudos
Not able to perform update in delta table in databricks using 3 tables
Hi,I am able to perform merge from 2 tables but have requirement to update table based on 3 tables like following query.update a set a.name=b.namefrom table1 a inner join table2 b on a.id=b.idinner join table3 c on a.id=c.idThanks in advance..
- 1606 Views
- 3 replies
- 0 kudos
- 0 kudos
Hi @upendra kumar sharma​ Help us build a vibrant and resourceful community by recognizing and highlighting insightful contributions. Mark the best answers and show your appreciation!Thanks and Regards
- 0 kudos
- 2825 Views
- 4 replies
- 0 kudos
Error in SQL statement: AnalysisException: The query operator `UpdateCommandEdge` contains one or more unsupported expression types Aggregate, Window or Generate.
com.databricks.backend.common.rpc.DatabricksExceptions$SQLExecutionException: org.apache.spark.sql.AnalysisException: The query operator `UpdateCommandEdge` contains one or more unsupportedexpression types Aggregate, Window or Generate.Invalid expres...
- 2825 Views
- 4 replies
- 0 kudos
- 0 kudos
Hi @Pradeep Chauhan​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answe...
- 0 kudos
- 2310 Views
- 4 replies
- 3 kudos
Is the Model tab not available in Databricks Community cloud edition?
I am currently doing the Scalable Machine Learning Course and I observed that the menu options available in the videos are a bit different from what I have on my edition.Is it because I am using the Community edition or there is some setting to make ...
- 2310 Views
- 4 replies
- 3 kudos
- 3 kudos
Hi @Pradeep Gadkari​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answe...
- 3 kudos
- 3709 Views
- 4 replies
- 0 kudos
Slow updates/upserts in Delta tables
When using Delta tables with DBR jobs or even with DLT pipelines, the upserts (especially updates) (on key and timestamp) are taking quite higher than expected time to update the files/tables data (~2 mins for even 1 record poll) (Inserts are lightni...
- 3709 Views
- 4 replies
- 0 kudos
- 0 kudos
Hi @Surya Agarwal​ Hope everything is going great.Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us so...
- 0 kudos