Automate data bricks batch which copy data from file/database and put it in some other file/database
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
07-02-2025 07:14 AM
Hi,
I have created a data bricks batch which copy data from a file/database table and put it in some other file/database table.
I want to automate the functional test cases of this data brick batch.
Can someone suggest, is there any tool available for this kind of job or any approach that can help me?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
07-02-2025 10:39 AM
Create a workflow for your batch load notebook and then schedule it.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
07-02-2025 09:18 PM
I want to keep my automation framework outside the data bricks environment.
is there any way where I can update the record to parquet files/Unity catalog/Delta tables from my framework.
Also start the data bricks batch from outside framework as well.
Please suggest