Performance issue when reading data from a view.
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
01-20-2025 07:49 AM
Hi All,
We are facing some performance issue and I need your help to know what could be the best approach to follow here.
Existing:
- For each region, we have view(Reg1_View,Reg2_View..) to pull data from table( we don't have direct access to table).
- And then we are combining data from all view into another view named All_Reg_View(Reg1_View union all Reg2_View..).
- Using the combining view All_Reg_View in join condition with other view in notebook and create target delta table.
All views are constructed same way and we are dealing with billions of data.
Proposed approach:
- Create one staging table(truncate & load) replacing the combined view(All_Reg_View) by performing union of all views.
- Partition the stg table based on region.
- Use the stg tables in joining to create target delta table.
Please let me know your thought.