by
sanjay
• Valued Contributor II
- 7348 Views
- 0 replies
- 0 kudos
Hi,I have data pipeline which is running continuously, processes the micro batch data and store data in delta lake. This is taking care of any new data.But at times, I need to process historical data without disturbing real time data processing.Is th...
- 7348 Views
- 0 replies
- 0 kudos
by
laksh
• New Contributor II
- 1062 Views
- 2 replies
- 0 kudos
I was wondering how the Unity Catalog would help in data quality validations for real time (streaming data) data ingestion.
- 1062 Views
- 2 replies
- 0 kudos
Latest Reply
Hi @arun laksh​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Thanks!
1 More Replies
- 581 Views
- 0 replies
- 0 kudos
Hello,I have a table on Azure Databricks that I keep updating with the "A" notebook.And I want to real time plotting the query result from the table (let's say SELECT COUNT(name), name FROM my_schema.my_table GROUP BY name).I know about Azure Applica...
- 581 Views
- 0 replies
- 0 kudos
- 3953 Views
- 12 replies
- 9 kudos
Delta lake have 3 levels to maintain data quality ( bronze , silver and gold tables ) but this supports the reporting and BI solutions how does this supports the streaming analytics ?example : I have an app that loads all the operational data in adls...
- 3953 Views
- 12 replies
- 9 kudos
Latest Reply
@Basavaraj Angadi​ , Just a friendly follow-up. Do you still need help, or @Werner Stinckens​ 's response help you to find the solution? Please let us know.
11 More Replies
- 3340 Views
- 9 replies
- 12 kudos
I'm trying to query delta tables using JDBC connector in a Ruby app. I've noticed that it takes around 8 seconds just to connect with databricks cluster and then additional time to run the query.The app is connected to a web portal where users genera...
- 3340 Views
- 9 replies
- 12 kudos
Latest Reply
Hi @Aman Sehgal​ Could you please check SQL endpoints? SQL endpoint uses a photon engine. It can reduce the query processing time. And Serverless SQL endpoint can accelerate the launch timemore info: https://docs.databricks.com/sql/admin/sql-endpoin...
8 More Replies