- 3129 Views
- 2 replies
- 0 kudos
time travel with DLT
Needed some help with Time Travel with Delta Live tables We were trying to figure out if we can go in and alter the history on this table, and what would happen to data that we mass upload? By this we mean we have data from the past that we would ...
- 3129 Views
- 2 replies
- 0 kudos
- 0 kudos
Delta Live Tables leverage Delta Lake, or Delta Tables. Delta tables, through transactions (e.g. insert, update, delete, merges, optimization) create versions of said Delta Table. Once a version is created it cannot be altered, it is immutable. Yo...
- 0 kudos
- 2807 Views
- 1 replies
- 0 kudos
Upgrade Spark version 3.2 to 3.4+
Hi Team,We would like to upgrade from Spark version 3.2 to 3.4+ (Databricks Runtime - 10.4 to 12.2/13.3)We would like to understand how complex upgradation is this and challenges which we face? what are the technical steps and precautions we need to ...
- 2807 Views
- 1 replies
- 0 kudos
- 2112 Views
- 0 replies
- 0 kudos
Customer Managed Keys in Databricks (AWS)
Hi Databricks Team,Could you please provide me the detailed steps on how to be enabled customer managed keys in databricks (AWS) Account, if there is any video on it that would be great helpful.Regards,Phanindra
- 2112 Views
- 0 replies
- 0 kudos
- 4427 Views
- 0 replies
- 0 kudos
Spark Streaming Issues while performing left join
Hi team,I'm struck in a Spark Structured streaming use-case.Requirement: To read two streaming data frames, perform a left join on it and display the results. Issue: While performing a left join, the resultant data frame contains only rows where ther...
- 4427 Views
- 0 replies
- 0 kudos
- 1123 Views
- 0 replies
- 0 kudos
AI uses
Delve into the transformative realm of AI applications, where innovation merges seamlessly with technology's limitless possibilities.Explore the multifaceted landscape of AI uses and its dynamic impact on diverse industries at StackOfTuts.
- 1123 Views
- 0 replies
- 0 kudos
- 1915 Views
- 1 replies
- 0 kudos
Load GCP data to Databricks using R
I'm working with Databricks and Google Cloud in the same project. I want to load specific datasets stored in GCP into a R notebook in Databricks. I currently can see the datasets in BigQuery. The problem is that using the sparklyr package, I'm not ab...
- 1915 Views
- 1 replies
- 0 kudos
- 3116 Views
- 0 replies
- 1 kudos
Best AI Art Generator
AI art generator uses artificial intelligence to create captivating artworks, redefining the boundaries of traditional creativity and enabling endless artistic possibilities.AI photo restoration is a groundbreaking technology that employs artificial ...
- 3116 Views
- 0 replies
- 1 kudos
- 2673 Views
- 1 replies
- 0 kudos
Multi Customer setup
We are trying to do POC to have shared resource like compute across multiple customer, Storage will be different, Is this possible ?
- 2673 Views
- 1 replies
- 0 kudos
- 6454 Views
- 0 replies
- 0 kudos
Alter table
Hi Team, Could you please suggest, Do we have an alternate approach to alter the table instead of creating a new table and copying the data as part of the deployment.
- 6454 Views
- 0 replies
- 0 kudos
- 2433 Views
- 1 replies
- 0 kudos
Constraint options usage
I am trying to use the constraints options:NOT ENFORCEDDEFERRABLEINITIALLY DEFERREDNORELYHowever it seems I am not able to use them successfully. When I try to use them with PRIMARY KEYS (not sure if it is possible), I am not able to enforce any key....
- 2433 Views
- 1 replies
- 0 kudos
- 0 kudos
BTW the forum is bugged I can't paste code..
- 0 kudos
- 2462 Views
- 1 replies
- 0 kudos
Resolved! Databricks Certification Exam Got Suspended. Need help in resolving the issue
Hi @Cert-Team ,My Databricks exam got suspended on December 9, 2023, at 11:30, and it is still in the suspended state.During the exam, it was initially paused due to poor lighting, but after addressing that, it worked fine. However, after some time, ...
- 2462 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @Jay_adb I'm sorry to hear you had this issue. Thanks for filing a ticket with the support team. I have sent a message to them to look into your ticket and resolve asap.
- 0 kudos
- 2019 Views
- 0 replies
- 2 kudos
DAB "bundle deploy" Dry Run
Is there a way to perform a dry-run with "bundle deploy" in order to see the job configuration changes for an environment without actually deploying the changes?
- 2019 Views
- 0 replies
- 2 kudos
- 8716 Views
- 0 replies
- 1 kudos
🌟 End-of-Year Community Survey 🌟
Hello Community Members, We value your experience and want to make it even better! Help us shape the future by sharing your thoughts through our quick Survey. Ready to have your voice heard? Click here and take a few moments to complete the surv...
- 8716 Views
- 0 replies
- 1 kudos
- 2571 Views
- 1 replies
- 0 kudos
Multi Task Job creation through Pulumi
I am trying to create a multi-task Databricks Job in Azure Cloud with its own cluster.Although I was able to create a single task job without any issues, the code to deploy the multi-task job fails due to the following cluster validation error:error:...
- 2571 Views
- 1 replies
- 0 kudos
- 0 kudos
Hello @Retired_mod, thanks for your answer, but the problem keeps the same. I had already tested with different cluster configurations, single-node and multi-node, including those cluster configurations which worked with single task jobs, but the err...
- 0 kudos
- 1266 Views
- 0 replies
- 0 kudos
df.queryExecution.redactedSql is not working with Spark sql Listener
We are trying to capture the query executed by spark .We are trying to use df.queryExecution.redactedSql to get the SQL from query execution but it is not working in sqlListener
- 1266 Views
- 0 replies
- 0 kudos
