DLT Notebook and Pipeline Separation vs Consolidation

[ChristianRRL]

Super basic question. For DLT pipelines I see there's an option to add multiple "Paths". Is it generally best practice to completely separate `bronze` from `silver` notebooks? Or is it more recommended to bundle both raw `bronze` and clean `silver` data into notebooks together and separate notebooks by something else?

Similarly, what is an "appropriate" number of tables to process in a single DLT pipeline? E.g. I'm already processing 20 tables from one source and because they're all bronze I have to squint (or realistically just zoom in on the DLT Graph). These 20 bronze tables would flow into 20 silver tables for this current example, but for separate notebook (+maybe separate DLT pipeline) I could see 100s of bronze tables flowing into a few distinct silver tables.

[Lakshay · Accepted solution]

You can run as many tables as you want provided the cluster capacity. Also, if you are processing large no. of tables, using list view in DLT might be a better option as compared to graph view.

[ChristianRRL]

This is great! I completely missed the list view before.