- 9799 Views
- 5 replies
- 5 kudos
How to handle java.io.Exception in python notebook
I'm attempting to mount a volume using dbutils.fs.mount in a python workbookin the exception handling for this statement, I have found an exception that doesn't get caught using the standard try/except handlingfor example, if passing through a contai...
- 9799 Views
- 5 replies
- 5 kudos
- 5 kudos
Hi @Stuart Parker​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Than...
- 5 kudos
- 2447 Views
- 0 replies
- 0 kudos
{{start_time}} isn't accurate and doesn't behave logically for multi-task jobs
I am trying to run an incremental data processing job using python wheel.The job is scheduled to run e.g. every hour.For my code to know what data increment to process, I inject it with the {{start_time}} as part of the command line, like so["end_dat...
- 2447 Views
- 0 replies
- 0 kudos
- 7387 Views
- 7 replies
- 7 kudos
Forgot password email is not sent
I couldn't login to my Databricks premium 14 day free trial workspace so I tried to reset my password but the email was never sent. Please help - I've already lost a day of my trial because I couldn't login. Thank you
- 7387 Views
- 7 replies
- 7 kudos
- 7 kudos
Hi @Sarah Rudston​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Than...
- 7 kudos
- 3452 Views
- 2 replies
- 2 kudos
Resolved! Can someone give me a hand with Azure quotas and Databricks?
I'm trying to use databricks learning and go through the various lessons using my Azure pay-as-you-go account. IT's been super frustrating because I'm constantly hitting quota limits. First it was Regional vCPUs, then dedicated, now 'low priority' ...
- 3452 Views
- 2 replies
- 2 kudos
- 2 kudos
Hi @user_bfe431b2 erithtotl​ Does @Hubert Dudek​ and @Kaniz Fatma​ 's response answer your question? If yes, would you be happy to mark it as best so that other members can find the solution more quickly?We'd love to hear from you.Thanks!
- 2 kudos
- 7964 Views
- 5 replies
- 1 kudos
Write in Single CSV file
We are reading 520GB partitions files from CSV and when we write in a Single CSV using repartition(1) it is taking 25+ hours. please let us know an optimized way to create a single CSV file so that our process could complete within 5 hours.
- 7964 Views
- 5 replies
- 1 kudos
- 1 kudos
Hi @mohit kumar suthar​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you...
- 1 kudos
- 3132 Views
- 4 replies
- 1 kudos
Hi,On the 27 of October, I did and pass my exam for data bricks data engineer associate but I didn't get my badge. could you help me with that? Al...
Hi,On the 27 of October, I did and pass my exam for data bricks data engineer associate but I didn't get my badge. could you help me with that? Also, I raised the ticker number: #00232560 but also I don't get any updates about it.cc:@Jose Gonzalez​
- 3132 Views
- 4 replies
- 1 kudos
- 1 kudos
Hi @SAUL Ramirez​ Hope everything is going great.Just wanted to check in if you were able to resolve your issue.If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please visit the cr...
- 1 kudos
- 8872 Views
- 6 replies
- 5 kudos
- 8872 Views
- 6 replies
- 5 kudos
- 5 kudos
Could you please confirm if you are using E2 architecture (https://docs.databricks.com/getting-started/overview.html#e2-architecture) or is it a single tenant shard?
- 5 kudos
- 2761 Views
- 1 replies
- 7 kudos
All-purpose compute clusters that are attached to a pool are no longer able to switch to a different pool/change to a non-pool worker/driver.
Would like to know if anyone else is experiencing this - we're seeing this across 5+ different Databricks workspaces in both AWS and Azure.Reproduction: Create all purpose compute cluster, attach it to existing pool, save and start cluster. Edit clus...
- 2761 Views
- 1 replies
- 7 kudos
- 7 kudos
We're also seeing the same behavior when trying to change the pool on an all-purpose cluster using Terraform and Databricks Labs Terraform provider as well. The Terraform apply will go through and say the cluster was updated to the new pool id, but t...
- 7 kudos
- 12079 Views
- 9 replies
- 4 kudos
Resolved! Error While creating the Databricks workspace on the AWS cloudformation stack
I am getting the following error in databricksAPIFunction resource creation and AWS Stack is failing with the rollback. Resource handler returned message: "Your access has been denied by S3, please make sure your request credentials have permission t...
- 12079 Views
- 9 replies
- 4 kudos
- 4 kudos
Hi @Kumar Shanu​,Thank you for coming back and letting us know.It was really great of you to mark an answer as best and for pointing everyone in the right direction.Have a great Databricks journey ahead!
- 4 kudos
- 2452 Views
- 0 replies
- 7 kudos
accounts.cloud.databricks.com
How to Enable Features in Data bricks to explicitly opt-in As a Databricks user, sometimes we may want to use account features that are supported for all workspaces under that account. once we enable them, then users from workspaces will be able to u...
- 2452 Views
- 0 replies
- 7 kudos
- 9207 Views
- 4 replies
- 4 kudos
Bootstrap Timeout during cluster start on AWS cloud
Sometimes am getting the below error when the cluster is started. Have attached the system log of the below mentioned instance from AWS. In recent days am getting this error for very frequently. Have seen same error is reported earlier and it marked ...
- 9207 Views
- 4 replies
- 4 kudos
- 4 kudos
@Kaniz Fatma​ @karthik p​ Since morning, am facing the issue again. Seems the issue is intermittent and it fails the pipeline in mid of the ETL process. Couldn't able to get the exact root cause of the issue. Can someone provide what would be the wo...
- 4 kudos
- 9971 Views
- 7 replies
- 9 kudos
How are people developing python libraries for use within a team on databricks?
In the past, before databricks, I would try and pull commonly used functions and features out of notebooks and save them in a python library that the whole team would work on and develop. This allowed for good code reuse and maintaining best practic...
- 9971 Views
- 7 replies
- 9 kudos
- 9 kudos
The way we do this is to package as much re-usable code up into a common library as possible and then test it to within an inch of it's life with unit tests (I tend to use unittest for lower barrier to entry, but which ever framework works best for y...
- 9 kudos
- 10998 Views
- 1 replies
- 6 kudos
Resolved! Databricks Notebook Export error - Error: InvalidConfigurationError: You haven't configured the CLI yet! Please configure by entering `/databricks/python3/bin/databricks configure`
I have Databricks notebook which have some SQL code. I want to export it to Ipython notebook. Export should be automated and I am using below command%shdatabricks workspace export --overwrite --format JUPYTER "/Users/USERNAME/SQL_Automate" /Users/USE...
- 10998 Views
- 1 replies
- 6 kudos
- 6 kudos
Hi @Abhishek Jain​ ,the error says that you need to configure databricks cli, by executing: databricks configurehttps://docs.databricks.com/dev-tools/cli/index.html#set-up-authenticationRunning the command with configure option will create .databrick...
- 6 kudos
- 1363 Views
- 0 replies
- 0 kudos
Check consistency of global temp views
Currently we are facing an issue with global temp views. We create in one notebook a global temp view and query the results in another notebook on the same cluster. The issue is, that the result is not consistent. When we query the view several times...
- 1363 Views
- 0 replies
- 0 kudos
- 7825 Views
- 5 replies
- 8 kudos
Resolved! Change schema when writing to the Delta format
Is it possible to reapply schema in delta files? For example, we have a history with field string but from some point, we need to replace string with struct.In my case merge option and overwrite schema don't work.
- 7825 Views
- 5 replies
- 8 kudos
| User | Count |
|---|---|
| 1644 | |
| 793 | |
| 581 | |
| 349 | |
| 287 |
