Questions about the design of bronze, silver, and gold for live streaming pipelines

rt-slowth
Contributor

I'm envisioning a live streaming pipeline.
The bronze, or data ingestion, is being fetched using the directory listing mode of the autoloader.
I'm not using File Notification Mode because I detect about 2-300 data changes per hour.
I'm thinking about implementing silver and gold tables later.
I tried to implement silver and gold as streaming tables, but it was not easy.


Do you usually implement silver and gold as a table or materialized view?

Also, can you introduce how to optimize when putting a large amount of data into redshift?