- 14334 Views
- 8 replies
- 4 kudos
Cluster Scoped init script through pulumi
I am trying to run a cluster-scoped init script through Pulumi. I have referred to this documentation https://learn.microsoft.com/en-us/azure/databricks/clusters/configure#spark-configuration However, looks like the documentation is not very clear.I ...
- 14334 Views
- 8 replies
- 4 kudos
- 4 kudos
Hi @Sulfikkar Basheer Shylaja , Why don't you store the init-script on DBFS and just pass the dbfs:/ path of the init script in Pulumi? You could just run this code on a notebook-%python dbutils.fs.put("/databricks/init-scripts/set-private-pip-repos...
- 4 kudos
- 7288 Views
- 0 replies
- 0 kudos
Datbricks Exam Cancelled
I was giving my Databricks Certified Data Engineer Associate (Version 2) exam scheduled today at 11:15 AM but it got cancelled in between. What to do now?Please help me with this so that I can complete it.
- 7288 Views
- 0 replies
- 0 kudos
- 1597 Views
- 0 replies
- 1 kudos
Databricks Exam got Cancelled
I was giving my Databricks Certified Data Engineer Associate (Version 2) exam scheduled today at 11:15 AM but it got cancelled in between. What to do now?Please help me with this so that I can complete it.
- 1597 Views
- 0 replies
- 1 kudos
- 1747 Views
- 0 replies
- 1 kudos
Hello Everyone, I am thrilled to announce that we have our 4th winner for the raffle contest - @MUHAMMET EMIN KOSEOGLU . Please join me in congratu...
Hello Everyone,I am thrilled to announce that we have our 4th winner for the raffle contest - @MUHAMMET EMIN KOSEOGLU . Please join me in congratulating him on this remarkable achievement!Your dedication and hard work have paid off, and we are del...
- 1747 Views
- 0 replies
- 1 kudos
- 3117 Views
- 1 replies
- 1 kudos
Connection Error DataGrip Databricks
When trying to connect DataGrip with Databricks SQL, I'm able to do it for multiple connections by using a Token. However, for one specific connection it is not working. We internally tried everything, but we are not able to connect. Below is the err...
- 3117 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi, This looks like, the requested resource is forbidden. Could you please check the destination webserver and recheck the configuration? Please tag @Debayan with your next comment so that I will get notified. Thank you!
- 1 kudos
- 12614 Views
- 12 replies
- 0 kudos
Resolved! Can't use job cluster for scheduled jobs ADD_NODES_FAILED : Failed to add 9 containers to the cluster. Will attempt retry: false. Reason: Azure Quota Exceeded Exception
Hi everyone,I've been using my all purpose cluster for scheduled jobs and I've been told that it's a suboptimal thing to do and that using a job cluster for the scheduled jobs cuts costs by half.Unfortunately, when I tried to switch clusters on my ex...
- 12614 Views
- 12 replies
- 0 kudos
- 0 kudos
@Bassem Jaber If you are seeing same error then you need to increase quota, for that your azure plan should be changed from pay as you go to other plan. as pay-as-go azure model has limitations on quota increase
- 0 kudos
- 5445 Views
- 3 replies
- 0 kudos
How to trigger alert for twice per day at set times?
I need to create a databricks alert for 9:30am and 5pm every day...is there a way to do this with one alert? I can't use "Refresh every 1 day at time..." since this will only trigger once per day. I also can't use "Refresh every 12 hours at minute....
- 5445 Views
- 3 replies
- 0 kudos
- 122175 Views
- 15 replies
- 1 kudos
- 122175 Views
- 15 replies
- 1 kudos
- 1 kudos
This answer from Stackoverflow is spot on. https://stackoverflow.com/a/72500870/1290485
- 1 kudos
- 9740 Views
- 8 replies
- 6 kudos
Job aborted due to stage failure: Task 1863 in stage 10.0 failed 4 times, most recent failure: Lost task 1863.3 in stage 10.0 (TID 2021) (10.0.4.7 executor 2): org.apache.spark.SparkException: Python worker exited unexpectedly (crashed): Fatal Python erro
I am getting below error some time run my databricks notebook from ADF, If the executor node is one then it works fine, if it increases 2 or more some times its failing on same data.Cluster Detail : Standard_F4s_v2 · Workers: Standard_F4s_v2 · 1-8 wo...
- 9740 Views
- 8 replies
- 6 kudos
- 6 kudos
Hi @Ancil P A Can you give paste the complete stacktrace from the failed task (from failed stage 10.0) and the code snippet that you are trying to run in the notebook . Also, do you think you can raise a databricks support ticket for the same?
- 6 kudos
- 3910 Views
- 1 replies
- 0 kudos
Loading multiple gz files from ADLS to Delta Lake/Delta table in ADB
I have several gz files (file.csv.gz) in a ADLS folder which are of pretty big size. All of these files are extractd from the same base table so it has the similar data but of different dates. How can I transfer them in delta lake/delta table. We wou...
- 3910 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi, You can read GZ files through spark. https://stackoverflow.com/questions/42761912/how-to-read-gz-compressed-file-by-pysparkPlease let us know if this helps. Also, please tag @Debayan with your next comment so that I will get notified. Thank you!
- 0 kudos
- 8122 Views
- 3 replies
- 0 kudos
How to filter the Spark UI for a notebook
When running spark under yarn each script has it's own self contained set of logs:- In databricks all I see if a list of jobs and stages that have been run on the cluster:- From a support perspective this is a nightmare.How can notebooks logs be grou...
- 8122 Views
- 3 replies
- 0 kudos
- 0 kudos
Hi @Dean Lovelace Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers...
- 0 kudos
- 14675 Views
- 4 replies
- 0 kudos
Function returns UNSUPPORTED_CORRELATED_SCALAR_SUBQUERY
Hello experts. The below function in Databricks gives UNSUPPORTED_CORRELATED_SCALAR_SUBQUERY error. We didn't have this issue though in Oracle. Is this a limitation of Databricks? Just to note the final result returns only one row. Thank you in advan...
- 14675 Views
- 4 replies
- 0 kudos
- 0 kudos
Hello @Suteja Kanuri can we have any respond on the above?Thank you.
- 0 kudos
- 1989 Views
- 1 replies
- 0 kudos
community.cloud.databricks.com
https://community.cloud.databricks.com/Last night I am still able to use it. This morning breaks down totally, and I could not login. Please help.
- 1989 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @THIAM HUAT TAN Thank you for reaching out, and we’re sorry to hear about this log-in issue! We have this Community Edition login troubleshooting post on Community. Please take a look, and follow the troubleshooting steps. If the steps do not res...
- 0 kudos
- 13760 Views
- 5 replies
- 4 kudos
Resolved! Databricks Certification Voucher Code not received
I have an exam to take for the Databricks Associate ML certification early this week. I have raised multiple tickets for the same and previously but didn't receive any reply on the same. I have attended the webinar for 3 days on Data Engineering as m...
- 13760 Views
- 5 replies
- 4 kudos
- 4 kudos
Hi @Avinash Tiwari I hope you are doing great!I have forwarded your query to our Academic team. Soon the problem will be resolved. Please bear with us.Thanks and Regards
- 4 kudos
- 18641 Views
- 3 replies
- 1 kudos
Root cause analysis and Woraround for "Error: AttributeError: type object 'Retry' has no attribute 'DEFAULT_METHOD_WHITELIST'&quo...
Root cause analysis and Woraround for "Error: AttributeError: type object 'Retry' has no attribute 'DEFAULT_METHOD_WHITELIST'"---------------------------------------------ProblemWhen using databricks-cli, starting with 4th of May this error occurs:Er...
- 18641 Views
- 3 replies
- 1 kudos
- 1 kudos
For what its worth I can confirm the pinning works. I also pinned the requests package. See my details here as I am using pipelines to handle deployment. I've back linked this issue on the databricks website to this item as well.https://github.co...
- 1 kudos
| User | Count |
|---|---|
| 1645 | |
| 793 | |
| 584 | |
| 349 | |
| 287 |
