https://community.databricks.com/t5/get-started-discussions/dlt-notebook-and-pipeline-separation-vs-consolidation/m-p/57848#M2281 <P>This is great! I completely missed the list view before.</P> Thu, 18 Jan 2024 23:54:28 GMT ChristianRRL 2024-01-18T23:54:28Z https://community.databricks.com/t5/get-started-discussions/dlt-notebook-and-pipeline-separation-vs-consolidation/m-p/57798#M2277 <P>Super basic question. For DLT pipelines I see there's an option to add multiple "Paths". Is it generally best practice to completely separate `bronze` from `silver` notebooks? Or is it more recommended to bundle both raw `bronze` and clean `silver` data into notebooks together and separate notebooks by something else?</P><P>Similarly, what is an "appropriate" number of tables to process in a single DLT pipeline? E.g. I'm already processing 20 tables from one source and because they're all bronze I have to squint (or realistically just zoom in on the DLT Graph). These 20 bronze tables would flow into 20 silver tables for this current example, but for separate notebook (+maybe separate DLT pipeline) I could see 100s of bronze tables flowing into a few distinct silver tables.</P><P><span class="lia-inline-image-display-wrapper lia-image-align-inline" image-alt="ChristianRRL_1-1705597040187.png" style="width: 749px;"><img src="https://community.databricks.com/t5/image/serverpage/image-id/5864i82DAA7B7D8D53D44/image-dimensions/749x148/is-moderation-mode/true?v=v2" width="749" height="148" role="button" title="ChristianRRL_1-1705597040187.png" alt="ChristianRRL_1-1705597040187.png" /></span></P> Thu, 18 Jan 2024 17:00:53 GMT https://community.databricks.com/t5/get-started-discussions/dlt-notebook-and-pipeline-separation-vs-consolidation/m-p/57798#M2277 ChristianRRL 2024-01-18T17:00:53Z https://community.databricks.com/t5/get-started-discussions/dlt-notebook-and-pipeline-separation-vs-consolidation/m-p/57825#M2279 <P>You can run as many tables as you want provided the cluster capacity. Also, if you are processing large no. of tables, using list view in DLT might be a better option as compared to graph view.</P> Thu, 18 Jan 2024 19:27:41 GMT https://community.databricks.com/t5/get-started-discussions/dlt-notebook-and-pipeline-separation-vs-consolidation/m-p/57825#M2279 Lakshay 2024-01-18T19:27:41Z https://community.databricks.com/t5/get-started-discussions/dlt-notebook-and-pipeline-separation-vs-consolidation/m-p/57848#M2281 <P>This is great! I completely missed the list view before.</P> Thu, 18 Jan 2024 23:54:28 GMT https://community.databricks.com/t5/get-started-discussions/dlt-notebook-and-pipeline-separation-vs-consolidation/m-p/57848#M2281 ChristianRRL 2024-01-18T23:54:28Z