When dealing with B2B data building, the process of updating and managing your data can present unique challenges. Since your data updates involve new folders with files and you need to process the entire new folder, the concept of incremental processing might not directly apply to your situation. Running a full refresh every time could indeed be necessary, as you're effectively processing new chunks of data each time. This approach, while not as efficient as incremental updates often used in B2B data building, might be the most straightforward given your data source's behavior.

In the realm of B2B data building, tools like Delta Live Tables and Delta Lake can still be highly relevant in this scenario, especially if you're working with large volumes of data or require features like ACID transactions and schema evolution. These technologies offer a robust framework for managing complex data workflows. However, since your data updates aren't truly incremental, some of the optimization benefits of DLTs might not be fully utilized in your specific context. Nonetheless, leveraging DLTs can still provide valuable benefits like improved data organization, versioning, and metadata management, which are critical aspects of maintaining high-quality B2B data.