08-17-2021 12:57 PM
Specifically for write and read streaming data to HDFS or s3 etc. For IoT specific scenario how it performs on time series transactional data. Can we consider delta table as time series table?
09-20-2021 09:08 AM
Hi @Arindam Halder , Delta lake is more performant compared to a regular parquet table. pls check below for some stats on the performance
https://docs.azuredatabricks.net/_static/notebooks/delta/optimize-python.html
yes, you can use it for time series forecasting. You could watch the below video to see how forecasting is done at scale using Delta lake
you could also find the delta streaming APIs here
https://docs.databricks.com/delta/delta-streaming.html#table-streaming-reads-and-writes
09-02-2021 02:47 AM
Hi @ ArindamHalder! My name is Kaniz, and I'm a technical moderator here. Great to meet you, and thanks for your question! Let's see if your peers on the Forum have an answer to your questions first. Or else I will follow up shortly with a response.
09-20-2021 09:08 AM
Hi @Arindam Halder , Delta lake is more performant compared to a regular parquet table. pls check below for some stats on the performance
https://docs.azuredatabricks.net/_static/notebooks/delta/optimize-python.html
yes, you can use it for time series forecasting. You could watch the below video to see how forecasting is done at scale using Delta lake
you could also find the delta streaming APIs here
https://docs.databricks.com/delta/delta-streaming.html#table-streaming-reads-and-writes
04-04-2022 01:28 AM
Hi @Arindam Halder , How is it going?
Were you able to resolve your problem?
Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections.
Click here to register and join today!
Engage in exciting technical discussions, join a group with your peers and meet our Featured Members.