I second the recommendations: auto load with trigger, batch processing instead of continuous streaming where use case permits. In addition,

• test with a small batch first
• favor fewer larger workers over more smaller workers
• adjust your job cluster overtime, by looking at spark UI and cluster metrics to see where steps can be optimized and computing resources reduced