Hi ! The cost field under planning_information in your DLT (Delta Live Tables) workflow system event logs represents an estimated cost for the particular maintenance type operation. In this context, MAINTENANCE_TYPE_COMPLETE_RECOMPUTE indicates that ...
AQE applies to all queries that are:Non-streamingContain at least one exchange (usually when there’s a join, aggregate, or window), one sub-query, or both.Not all AQE-applied queries are necessarily re-optimized. The re-optimization might or might no...
Hi ! Given the issues you're facing with schema inference and long processing times, it might be helpful to preprocess the JSON files to ensure they are properly formatted and manageable. Here's a step-by-step approach to split the JSON files into s...