There will be no performance impact if you want to keep " Delta log retention to 3000". However, it will increase the storage cost so it's not advisable to use a large number until really needed for the business use cases.The default delta.logRetenti...
At present DLT does not support installing the maven library from the DLT pipeline. In the future this feature will come for sure so please wait for some time and keep checking data bricks runtime release docs https://docs.databricks.com/release-note...
Autoloader doesn't support changing the source path for running job so if you change your source path your stream fails because the source path has changed. However, if you really want to change the path you can change it by using the new checkpoint ...
Hi @Brad Barker​ , In the 1st run of the jar-based job when the 1st job/task starts executor fetches the jar/library from driver to executor. Which will take some time but in the 2nd run or any consecutive runs, there will be no fetch operation so al...
