https://community.databricks.com/t5/technical-blog/finops-and-databricks-system-tables/ba-p/127796 <H3 class="graf graf--h3">Original post published on 30 May 2024, <A href="https://caiomsouza.medium.com/finops-and-databricks-system-tables-afd38b9177e1" target="_self">here</A>.<BR /><BR />FinOps and Databricks System&nbsp;Tables</H3> <P class="graf graf--p">Dear all,</P> <P class="graf graf--p">I have to admit that FinOps and System Tables are not the most interesting topic to talk about it, but everything we need to put to production will generate a cost and it is important to monitor the usage, etc.</P> <P class="graf graf--p">Having said that, I would like to talk about FinOps and System Tables.</P> <H3 class="graf graf--h3">Introduction to Databricks System&nbsp;Tables</H3> <P class="graf graf--p">System Tables are a Databricks-hosted analytical store for operational and usage data.</P> <P class="graf graf--p">System Tables can be used for monitoring, analyzing performance, usage, and behavior of Databricks Platform components. By querying these tables, users can gain insights into how their jobs, notebooks, users, clusters, ML endpoints, and SQL warehouses are functioning and changing over time. This historical data can be used to optimize performance, troubleshoot issues, track usage patterns, and make data-driven decisions.</P> <P class="graf graf--p">Overall, System Tables provide a means to enhance observability and gain valuable insights into the operational aspects of Databricks usage, enabling users to better understand and manage their workflows and resources.</P> <P class="graf graf--p">Databricks System Tables offer a comprehensive solution for monitoring, analyzing, and optimizing the operational aspects of the Databricks platform. These tables are perfect for:</P> <P class="graf graf--p"><span class="lia-unicode-emoji" title=":bar_chart:">📊</span> Cost and Usage Analytics: Track and manage your expenses effectively.<BR /><span class="lia-unicode-emoji" title=":rocket:">🚀</span> Efficiency Analytics: Optimize performance and resource allocation.<BR /><span class="lia-unicode-emoji" title=":locked:">🔒</span> Audit Analytics: Ensure compliance and security with comprehensive auditing.<BR /><span class="lia-unicode-emoji" title=":scroll:">📜</span> GDPR Regulation: Maintain data privacy and adhere to regulatory standards.<BR /><span class="lia-unicode-emoji" title=":direct_hit:">🎯</span> Service Level Objective Analytics: Meet your SLAs with precision.<BR /><span class="lia-unicode-emoji" title=":white_heavy_check_mark:">✅</span> Data Quality Analytics: Ensure the integrity and quality of your data.</P> <P class="graf graf--p">By querying System Tables, gain valuable insights into jobs, notebooks, clusters, ML endpoints, and SQL warehouses. Harness this historical data to troubleshoot issues, track usage patterns, and make informed, data-driven decisions.</P> <P class="graf graf--p">&nbsp;</P> <FIGURE class="graf graf--figure"><span class="lia-inline-image-display-wrapper lia-image-align-inline" image-alt="caiomoreno_4-1754652410189.png" style="width: 400px;"><img src="https://community.databricks.com/t5/image/serverpage/image-id/18843iA9D00946D874CDBD/image-size/medium?v=v2&amp;px=400" role="button" title="caiomoreno_4-1754652410189.png" alt="caiomoreno_4-1754652410189.png" /></span> <P>&nbsp;</P> <FIGCAPTION class="imageCaption">System Tables Dashboards<BR /><BR /></FIGCAPTION> </FIGURE> <FIGURE class="graf graf--figure"><span class="lia-inline-image-display-wrapper lia-image-align-inline" image-alt="caiomoreno_5-1754652410188.png" style="width: 400px;"><img src="https://community.databricks.com/t5/image/serverpage/image-id/18844iDF57B919B800ED7C/image-size/medium?v=v2&amp;px=400" role="button" title="caiomoreno_5-1754652410188.png" alt="caiomoreno_5-1754652410188.png" /></span> <P>&nbsp;</P> </FIGURE> <H1 id="d707" class="ma mb fr bf mc md me mf mg mh mi mj mk ml mm mn mo mp mq mr ms mt mu mv mw mx bk" data-selectable-paragraph="">Try it now:</H1> <P class="pw-post-body-paragraph lc ld fr le b lf my lh li lj mz ll lm ln na lp lq lr nb lt lu lv nc lx ly lz fk bk" data-selectable-paragraph="">Here we have a Demo where you can use System Tables in your Databricks workspace.</P> <H1 id="be96" class="ma mb fr bf mc md me mf mg mh mi mj mk ml mm mn mo mp mq mr ms mt mu mv mw mx bk" data-selectable-paragraph="">System Tables: Billing Forecast, Usage Analytics, and Access Auditing With Databricks Unity Catalog</H1> <P class="pw-post-body-paragraph lc ld fr le b lf my lh li lj mz ll lm ln na lp lq lr nb lt lu lv nc lx ly lz fk bk" data-selectable-paragraph=""><A class="ag od" href="https://www.databricks.com/resources/demos/tutorials/governance/system-tables" target="_blank" rel="noopener ugc nofollow">https://www.databricks.com/resources/demos/tutorials/governance/system-tables</A><BR /><A class="ag od" href="https://notebooks.databricks.com/demos/uc-04-system-tables/index.html#" target="_blank" rel="noopener ugc nofollow">https://notebooks.databricks.com/demos/uc-04-system-tables/index.html#</A></P> <H1 id="ecb0" class="ma mb fr bf mc md me mf mg mh mi mj mk ml mm mn mo mp mq mr ms mt mu mv mw mx bk" data-selectable-paragraph="">Databricks Cost Analysis using Power BI Template</H1> <P class="pw-post-body-paragraph lc ld fr le b lf my lh li lj mz ll lm ln na lp lq lr nb lt lu lv nc lx ly lz fk bk" data-selectable-paragraph="">For the Power BI users, I would recommend this material</P> <P class="pw-post-body-paragraph lc ld fr le b lf lg lh li lj lk ll lm ln lo lp lq lr ls lt lu lv lw lx ly lz fk bk" data-selectable-paragraph="">Cost control and cross charge with power bi template on top of Databricks system tables<BR /><A class="ag od" href="https://community.databricks.com/t5/technical-blog/databricks-cost-analysis-and-cross-charge-with-power-bi/ba-p/59820" target="_blank" rel="noopener ugc nofollow">https://community.databricks.com/t5/technical-blog/databricks-cost-analysis-and-cross-charge-with-power-bi/ba-p/59820</A><BR /><A class="ag od" href="https://github.com/lipinght/Databricks-cost-analysis-using-powerbi" target="_blank" rel="noopener ugc nofollow">https://github.com/lipinght/Databricks-cost-analysis-using-powerbi</A></P> <P>Learn more:</P> <P class="pw-post-body-paragraph lc ld fr le b lf lg lh li lj lk ll lm ln lo lp lq lr ls lt lu lv lw lx ly lz fk bk" data-selectable-paragraph=""><A class="ag od" href="https://docs.databricks.com/en/admin/system-tables/index.html" target="_blank" rel="noopener ugc nofollow">https://docs.databricks.com/en/admin/system-tables/index.html</A><BR /><A class="ag od" href="https://learn.microsoft.com/en-us/azure/databricks/admin/system-tables/" target="_blank" rel="noopener ugc nofollow">https://learn.microsoft.com/en-us/azure/databricks/admin/system-tables/</A><BR /><A class="ag od" href="https://github.com/drcaiomoreno/databricks-finops-system-tables" target="_blank" rel="noopener ugc nofollow">https://github.com/drcaiomoreno/databricks-finops-system-tables</A></P> <P class="pw-post-body-paragraph lc ld fr le b lf lg lh li lj lk ll lm ln lo lp lq lr ls lt lu lv lw lx ly lz fk bk" data-selectable-paragraph="">Databricks Specialist Sessions (Observability and FinOps with Overwatch and more):<SPAN>&nbsp;</SPAN><A class="ag od" href="https://vimeo.com/869043852" target="_blank" rel="noopener ugc nofollow">https://vimeo.com/869043852</A></P> <H1 id="1617" class="ma mb fr bf mc md me mf mg mh mi mj mk ml mm mn mo mp mq mr ms mt mu mv mw mx bk" data-selectable-paragraph="">UCX</H1> <P class="pw-post-body-paragraph lc ld fr le b lf my lh li lj mz ll lm ln na lp lq lr nb lt lu lv nc lx ly lz fk bk" data-selectable-paragraph="">If you do not have Unity Catalog enabled, you will need Unity Catalog to use System Tables, I recommend you to use UCX to start your Unity Catalog adoption:</P> <P class="pw-post-body-paragraph lc ld fr le b lf lg lh li lj lk ll lm ln lo lp lq lr ls lt lu lv lw lx ly lz fk bk" data-selectable-paragraph=""><A class="ag od" href="https://github.com/databrickslabs/ucx" target="_blank" rel="noopener ugc nofollow">https://github.com/databrickslabs/ucx</A></P> <H3 class="graf graf--h3"><BR />Original post published on 30 May 2024, <A href="https://caiomsouza.medium.com/finops-and-databricks-system-tables-afd38b9177e1" target="_self">here</A>.</H3> Tue, 09 Sep 2025 12:52:26 GMT caiomoreno 2025-09-09T12:52:26Z https://community.databricks.com/t5/technical-blog/finops-and-databricks-system-tables/ba-p/127796 <P class="pw-post-body-paragraph lc ld fr le b lf lg lh li lj lk ll lm ln lo lp lq lr ls lt lu lv lw lx ly lz fk bk" data-selectable-paragraph="">Dear all,</P> <P class="pw-post-body-paragraph lc ld fr le b lf lg lh li lj lk ll lm ln lo lp lq lr ls lt lu lv lw lx ly lz fk bk" data-selectable-paragraph="">I have to admit that FinOps and System Tables are not the most interesting topic to talk about it, but everything we need to put to production will generate a cost and it is important to monitor the usage, etc.</P> <P class="pw-post-body-paragraph lc ld fr le b lf lg lh li lj lk ll lm ln lo lp lq lr ls lt lu lv lw lx ly lz fk bk" data-selectable-paragraph="">Having said that, I would like to talk about FinOps and Databricks System Tables.</P> <P class="pw-post-body-paragraph lc ld fr le b lf lg lh li lj lk ll lm ln lo lp lq lr ls lt lu lv lw lx ly lz fk bk" data-selectable-paragraph="">Original post published on 30 May 2024, <A href="https://caiomsouza.medium.com/finops-and-databricks-system-tables-afd38b9177e1" target="_self">here</A>.</P> Tue, 09 Sep 2025 12:52:26 GMT https://community.databricks.com/t5/technical-blog/finops-and-databricks-system-tables/ba-p/127796 caiomoreno 2025-09-09T12:52:26Z