- 7044 Views
- 3 replies
- 2 kudos
Need help skipping previously executed cells in a failed Databricks job calling a notebook with multiple SQL cells
In Azure databricks, I have a job that calls a notebook which has multiple cells with sql queries. In case of any cell fails and when we restart the databricks job then how to skip previous cell which already ran and start only from the failed cell? ...
- 7044 Views
- 3 replies
- 2 kudos
- 2 kudos
Hi @Sandip Rath Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers y...
- 2 kudos
- 11372 Views
- 2 replies
- 1 kudos
Resolved! How does Auto Loader ingest data?
I have recently completed the Data Engineering with Databricks v3 course on the Partner Academy. Some of the quiz questions have me mixed up.Specifically, I am wondering about this question from the "Build Data Pipelines with Delta Live Tables and Sp...
- 11372 Views
- 2 replies
- 1 kudos
- 1 kudos
Hi @Kenny Shaevel Hope everything is going great.Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us so...
- 1 kudos
- 2079 Views
- 1 replies
- 0 kudos
- 2079 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @Paweł Tomczyk Hope everything is going great.Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us so...
- 0 kudos
- 2650 Views
- 1 replies
- 0 kudos
Dataframes to Pandas conversion step is failing with exception ""java.lang.IndexOutOfBoundsException: index: 16384, length: 4 (expected: range(0, 16384))"
Dataframes to Pandas conversion step is failing with exception ""java.lang.IndexOutOfBoundsException: index: 16384, length: 4 (expected: range(0, 16384))", PFB screenshot for more details
- 2650 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @Vindhya D Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers you...
- 0 kudos
- 2664 Views
- 2 replies
- 0 kudos
Difference of V3 exam for Databricks Certified Data Engineer Associate, comparing with V2
Hi,Does anyone know what's the difference of V3 exam for Databricks Certified Data Engineer Associate, comparing with V2?Looks like there is no practice exam for V3?Which version covers more stuff?Thanks,h_aloha
- 2664 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi @Helen Morgen Thank you for reaching out! Please submit a ticket to our Training Team here: https://help.databricks.com/s/contact-us?ReqType=training and our team will get back to you shortly.
- 0 kudos
- 3256 Views
- 3 replies
- 0 kudos
Resolved! when trying to use %run to a notebook, I'm getting an error.
this error is occurring randomly. sometimes it resolves on its own. not sure what is the cause of the error. The notebook I'm sourcing runs flawlessly. I'm only trying to import dates from this notebook
- 3256 Views
- 3 replies
- 0 kudos
- 0 kudos
Hi @viswa p Hope everything is going great.Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us so we ca...
- 0 kudos
- 2369 Views
- 1 replies
- 0 kudos
Can sql result display more than 25 records per page?
Hi! I have a result table with 41 rows. What should I do to make all rows available on the same page?
- 2369 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @wenting_deng wenting_deng Hope everything is going great.Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, pleas...
- 0 kudos
- 5943 Views
- 2 replies
- 2 kudos
Resolved! Lot of write shuffle on optimize + ZORDER, is it normal?
Hi! I'm optimizing several Tb of partitioned data on ZSTD lvl 9.It surprises me the level of shuffle write, it could make sense because of ZORDER but I want to be sure that I'm not missing something, here is some context: Could I be missing something...
- 5943 Views
- 2 replies
- 2 kudos
- 2 kudos
Hi @Alejandro Martinez Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best an...
- 2 kudos
- 26122 Views
- 2 replies
- 2 kudos
Resolved! Upgrade Python version in cluster
Currently I am using the following cluster. It is using the default python version of 3.9.5 and I would like to update it to 3.10.1.0How to achieve this?
- 26122 Views
- 2 replies
- 2 kudos
- 2 kudos
Hi @Ayush Modi Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers yo...
- 2 kudos
- 19409 Views
- 4 replies
- 3 kudos
Relationship between Databricks Account and Azure resources
Hello,What is the relationship between Databricks Account (as described in [1]) and Azure resources? Is Databricks Account created per Azure account? Or per Azure tenant? Or maybe per subscription?[1] https://learn.microsoft.com/en-us/azure/databrick...
- 19409 Views
- 4 replies
- 3 kudos
- 3 kudos
Hi @Chris Nawara Hope everything is going great.Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us so ...
- 3 kudos
- 9419 Views
- 1 replies
- 0 kudos
Iteration - Pyspark vs Pandas
Hello. Could someone please explain why iteration over a Pyspark dataframe is way slower than over a Pandas dataframe?Pysparkdf_list = df.collect()for index in range(0, len(df_list )):.....Pandasdf_pnd = df.toPandas() for index, row in df_p...
- 9419 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @ELENI GEORGOUSI Hope everything is going great.Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us ...
- 0 kudos
- 4044 Views
- 1 replies
- 0 kudos
Running terraform plan; databricks throws a "Permission_Denied: Missing required permissions [View] on node with ID "1759335429158542"
To give you a little bit of background:We use terraform to deploy a resource group with multiple Azure services Terraform leverages an Azure Service Principal that has Owner rights to the Azure subscriptionThis way, databricks is also deployed. We al...
- 4044 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @Gent Reshtani Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers...
- 0 kudos
- 6788 Views
- 10 replies
- 2 kudos
Selfpaced course link for "Apache Spark developer associate "
Please where can i find the the "Apache Spark developer associate " selfpaced course ?Thanks in advance.
- 6788 Views
- 10 replies
- 2 kudos
- 2 kudos
Hi @youssef ansari Hope everything is going great.Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us s...
- 2 kudos
- 3561 Views
- 3 replies
- 1 kudos
Create a Dockerfile from Cluster
Is there a way to create a Dockerfile from Workspace A's cluster configurations and deploy that on a different different cluster in Workspace B?
- 3561 Views
- 3 replies
- 1 kudos
- 1 kudos
Hi @Kevin Kim Hope everything is going great.Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us so we ...
- 1 kudos
- 1530 Views
- 1 replies
- 0 kudos
- 1530 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @Bhupesh Aggarwal Hope everything is going great.Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us...
- 0 kudos
| User | Count |
|---|---|
| 1632 | |
| 791 | |
| 513 | |
| 349 | |
| 287 |
