Hi @aleksandra_ch, thank you very much for your response and for keeping me updated on the new control options.

I have one additional question after reviewing the different pieces of documentation.
Is it considered a best practice or standard approach to build pipelines using Spark SQL rather than PySpark?

I ask this because, when reading the documentation, features such as Liquid Clustering, partition management, and the control mechanism described in the page you shared are mostly documented and explained using SQL examples.

Thank you in advance for your clarification.