Spark Standalone Cluster Configuration (Spark 3.0.0)-
1 Master2 Workers (4 cores each)
I am using Airflow SparkSubmitOperator to submit the job to Spark Master in Cluster mode. There are multiple(~20) DAGs on airflow submitting jobs to Spark. These ...