- 2899 Views
- 2 replies
- 2 kudos
Resolved! Reduce downtime of Postgres table - JDBC overwrite job
I want to overwrite a Postgresql table transactionStats which is used by the customer facing dashboards.This table needs to be updated every 30 mins. I am writing a AWS Glue Spark job via JDBC connection to perform this operation.Spark dataframe writ...
- 2899 Views
- 2 replies
- 2 kudos
- 2 kudos
Hi @Siddharth Kanojiya We haven't heard from you since the last response from @werners (Customer) . Kindly share the information with us, and in return, we will provide you with the necessary solution.Thanks and Regards
- 2 kudos
- 13521 Views
- 2 replies
- 2 kudos
Resolved! AZURE_QUOTA_EXCEEDED_EXCEPTION - even with more than vCPUs than Databricks recommends
I am running this Delta Live Tables PoC from databricks-industry-solutions/industry-solutions-blueprintshttps://github.com/databricks-industry-solutions/pos-dltI have Standard_DS4_v2 with 28GB and 8 cores x 2 workers - so a total of 16 cores. This is...
- 13521 Views
- 2 replies
- 2 kudos
- 2 kudos
Hi @Prasenjit Biswas We haven't heard from you since the last response from @Jose Gonzalez . Kindly share the information with us, and in return, we will provide you with the necessary solution.Thanks and Regards
- 2 kudos
- 1687 Views
- 1 replies
- 2 kudos
This site has alot of issues....
I need to login and I can't to try Databricks..so you have a OAUTH issue...I cant try Databricks at all because the country icon doesn't work and it sends a URI issue from your front end to the back end..."Request_URI=&Geo_country_code=&Geo_country_i...
- 1687 Views
- 1 replies
- 2 kudos
- 2 kudos
Hi @Jessica Woods Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers...
- 2 kudos
- 12137 Views
- 7 replies
- 3 kudos
Detectron2 install
I have been struggling to install Detectron 2. I think it is an issue with Cuda. Any adivise?install!pip install -q "detectron2@git+https://github.com/facebookresearch/detectron2.git@e2ce8dc#egg=detectron2"Error:qm8vsdal/detectron2_85faeed5ce7945dbad...
- 12137 Views
- 7 replies
- 3 kudos
- 3 kudos
Hi @Joshua RobergeSorry for the inconvenience!Kindly review the solution offered by @Suteja Kanuri.
- 3 kudos
- 32804 Views
- 21 replies
- 21 kudos
Resolved! How to access a jar file stored in Databricks Workspace ?
Hi All,We have a couple of jars stored in a workspace folder.We are using init scripts to copy the jars in the workspace to the /databricks/jars path.The init scripts do not seem to be able to find the files. The scripts are failing saying the files ...
- 32804 Views
- 21 replies
- 21 kudos
- 21 kudos
Hi @Bharath Kumar Ramachandran You're welcome! I'm glad you found the link useful. I empathize with your hope that Databricks would consider adding this option. It's possible that Databricks will take user feedback into account when planning future ...
- 21 kudos
- 3913 Views
- 4 replies
- 5 kudos
To do Optimization on the real time delta table
Hi Team,We have few prod tables which are created in s3 bucket, that have grown now very large, these tables are getting real time data continuously from round the clock databricks workflows; we would like run the optimization commands(optimize, zord...
- 3913 Views
- 4 replies
- 5 kudos
- 5 kudos
Hi @Sriram Kumar We haven't heard from you since the last response from @Suteja Kanuri . Kindly share the information with us, and in return, we will provide you with the necessary solution.Thanks and Regards
- 5 kudos
- 15745 Views
- 7 replies
- 9 kudos
virtual environment on azure databricks compute cluster
I'm using Azure Databricks and I'd like to create a project virtual environment, persisted on a shared compute cluster. As the cluster is shared for many projects, it is necessary to have virtual environments if I want to execute code runs from withi...
- 15745 Views
- 7 replies
- 9 kudos
- 9 kudos
Hi @Joshua L We haven't heard from you since the last response from @Debayan Mukherjee , and I was checking back to see if her suggestions helped you.Or else, If you have any solution, please share it with the community, as it can be helpful to ot...
- 9 kudos
- 1986 Views
- 1 replies
- 3 kudos
How to execute requests later for a number of times that exceeds the Maximum concurrent runs?
I am trying to start the same Jobs multiple times using the python sdk's "run_now" command.If the number of requests exceeds the Maximum concurrent runs, the status of the run will be Skipped and the run will not be executed.Is there any way to queue...
- 1986 Views
- 1 replies
- 3 kudos
- 3 kudos
Hi, We do have a private preview feature which will be enabled shortly for queueing. Please tag me (@Debayan Mukherjee ) with your next update so that I will get notified.
- 3 kudos
- 3814 Views
- 3 replies
- 3 kudos
Databricks run-time 12.2 LTS drop function problem
I am getting an error about the `drop function of pyspark` at a cluster using 12.2 LTS. When I check the error I see spark solved that bug, see SPARK-42444. Also when I check maintenance updates page, I saw this solved issue included the Databricks R...
- 3814 Views
- 3 replies
- 3 kudos
- 3 kudos
Hi @Sevval Mehder Elevate our community by acknowledging exceptional contributions. Your participation in marking the best answers is a testament to our collective pursuit of knowledge.
- 3 kudos
- 2667 Views
- 2 replies
- 3 kudos
Resolved! Implement & Test DR Plan
Hi,Can you direct me to any documentation on how to implement and test Disaster Recovery for Databricks (PAAS) on Azure ?Thx & RgdsRamdas
- 2667 Views
- 2 replies
- 3 kudos
- 3 kudos
Hi @Ramdas Panicher Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answe...
- 3 kudos
- 2191 Views
- 1 replies
- 1 kudos
- 2191 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi @Ayush Raj Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers you...
- 1 kudos
- 6315 Views
- 3 replies
- 4 kudos
Resolved! How to checkout specific commit version via databricks UI
I have integrated gitlab with my azure databricks repo, I am able to push and pull commits from the databricks UI, I want to checkout to a specific commit version via databricks UI.Note: I am aware that via the gitlab i have checkout to specific vers...
- 6315 Views
- 3 replies
- 4 kudos
- 4 kudos
After getting more context on databricks repo in details,Currently databricks doesn't support checkout of repo to specific commit.databricks provides only limited functionality mentioned belowAdd a repo and connect remotely laterClone a repo connecte...
- 4 kudos
- 3274 Views
- 2 replies
- 2 kudos
Trigger.AvailableNow getting stuck when there is no event
Hi, I have several streaming jobs, however one of them uses the Trigger.AvailableNow. The issue is that it gets stuck when there is no events or finishes ingesting all events. The expected behavior would be the job being shutdown.I've already checked...
- 3274 Views
- 2 replies
- 2 kudos
- 2 kudos
Hi, the source is an S3 bucket using file notification with SQS.No errors or warns in the logs, the AvailableNow trigger just gets stuck.
- 2 kudos
- 2717 Views
- 1 replies
- 0 kudos
what is best practice to handle the concurrency issue in batch processing?
Normally, our ELT framework takes in batches one by one and loads the data into target tables. But if more than one batches come in at the same time, the framework will break due to the concurrency issue that multiple sources are trying to write the ...
- 2717 Views
- 1 replies
- 0 kudos
- 0 kudos
you can partition you table to avoid the changes of getting this exception.
- 0 kudos
- 1785 Views
- 1 replies
- 3 kudos
www.databricks.com
Attention Community! For a limited period, we are offering a generous 50% discount on training at the Data + AI Summit. Simply apply the code FLS4vop5ep during the registration process. Hurry, though, as this offer will expire on June 12, 2023. Don'...
- 1785 Views
- 1 replies
- 3 kudos
- 3 kudos
Thank you for sharing this @Juliet Wu!!!
- 3 kudos
| User | Count |
|---|---|
| 1644 | |
| 793 | |
| 553 | |
| 349 | |
| 287 |
