- 1146 Views
- 3 replies
- 3 kudos
Specifically for write and read streaming data to HDFS or s3 etc. For IoT specific scenario how it performs on time series transactional data. Can we consider delta table as time series table?
- 1146 Views
- 3 replies
- 3 kudos
Latest Reply
Hi @Arindam Halder​ , How is it going?Were you able to resolve your problem?
2 More Replies
- 4158 Views
- 7 replies
- 2 kudos
What happens if we change the logic for the delta live tables and we do an incremental update. Does the table get reset (refresh) automatically or would it only apply the logic to new incoming data? would we have to trigger a reset in this case?
- 4158 Views
- 7 replies
- 2 kudos
Latest Reply
Here is my finding on when to refresh (reset) the table: If it is a complete table all the changes would be apply automatically. If the table is incremental table, you need to do a manually reset (full refresh).
6 More Replies
by
Brose
• New Contributor III
- 9371 Views
- 10 replies
- 2 kudos
I am trying to create a delta table for streaming data, but I am getting the following error; Error in SQL statement: ParseException: mismatched input 'CREATE' expecting {<EOF>, ';'}(line 2, pos 0).My statement is as follows;%sqlDROP TABLE IF EXISTS ...
- 9371 Views
- 10 replies
- 2 kudos
Latest Reply
@Ambrose Walker​ - If Jose's answer resolved your issue, would you be happy to mark that post as best? That will help others find the solution more quickly.
9 More Replies
by
guruv
• New Contributor III
- 2433 Views
- 5 replies
- 2 kudos
HI,i have several delta tables on Azure adls gen 2 storage account running databricks runtime 7.3. there are only write/read operation on delta tables and no update/delete.As part of release pipeline, below commands are executed in a new notebook in...
- 2433 Views
- 5 replies
- 2 kudos
Latest Reply
the auto optimize is sufficient, unless you run into performance issues.Then I would trigger an optimize. This will generate files of 1GB (so larger than the standard size of auto optimize). And of course the Z-Order if necessary.The suggestion to ...
4 More Replies
- 1141 Views
- 2 replies
- 6 kudos
Have you checked out the new Bill Inmon Book, Building the Data Lakehouse? https://dbricks.co/3uxCXjO What were your thoughts if you read it?
- 1141 Views
- 2 replies
- 6 kudos
Latest Reply
The quality of the book depends on the audience IMO. For people who have no background in data warehousing it will be interesting to read. For the others the book is too general and descriptive. The 'HOW do you do x' is missing.
1 More Replies
- 1730 Views
- 2 replies
- 0 kudos
I have an existing data pipeline which looks like this: A small MySQL data source (around 250 GB) and data passes through Debezium/ Kafka / a custom data redactor -> to Glue ETL jobs and finally lands on Redshift, but the scale of the data is too sm...
- 1730 Views
- 2 replies
- 0 kudos
Latest Reply
Dan_Z
Honored Contributor
There is a lot in this question, so generally speaking I suggest you reach out to the sales team at Databricks. You can talk to a solutions architect who get into more detail. Here are my general thoughts having seen a lot of customer arch:Generally,...
1 More Replies
- 876 Views
- 1 replies
- 0 kudos
What's the best way to federate a query to delta lake or the databricks from presto sql without having to create external tables? PrestoSQL doesn't have access to S3. Can PrestoSQL be configured with jdbc driver or plugin?
- 876 Views
- 1 replies
- 0 kudos
Latest Reply
Hi @ MohitAnchlia! My name is Kaniz, and I'm a technical moderator here. Great to meet you, and thanks for your question! Let's see if your peers on the Forum have an answer to your questions first. Or else I will follow up shortly with a response.
