topic How to run multiple spark streaming application on databricks cluster? in Data Engineering https://community.databricks.com/t5/data-engineering/how-to-run-multiple-spark-streaming-application-on-databricks/m-p/20691#M14000 <P>I started working on databricks. I need to migrate few streaming jobs from Ambari to Databricks. I deployed one job using jar and it. is working fine. But when I deploy the second job I faced an error " multiple spark streaming context not allowed". Is it possible to run multiple spark streaming jobs on databricks cluster?</P><P></P> Thu, 24 Jun 2021 15:13:46 GMT User16790091296 2021-06-24T15:13:46Z How to run multiple spark streaming application on databricks cluster? https://community.databricks.com/t5/data-engineering/how-to-run-multiple-spark-streaming-application-on-databricks/m-p/20691#M14000 <P>I started working on databricks. I need to migrate few streaming jobs from Ambari to Databricks. I deployed one job using jar and it. is working fine. But when I deploy the second job I faced an error " multiple spark streaming context not allowed". Is it possible to run multiple spark streaming jobs on databricks cluster?</P><P></P> Thu, 24 Jun 2021 15:13:46 GMT https://community.databricks.com/t5/data-engineering/how-to-run-multiple-spark-streaming-application-on-databricks/m-p/20691#M14000 User16790091296 2021-06-24T15:13:46Z Re: How to run multiple spark streaming application on databricks cluster? https://community.databricks.com/t5/data-engineering/how-to-run-multiple-spark-streaming-application-on-databricks/m-p/20692#M14001 <P>You can run multiple streaming applications in databricks clusters. By default, this would run in the same&nbsp;<A href="https://spark.apache.org/docs/latest/job-scheduling.html#scheduling-within-an-application" alt="https://spark.apache.org/docs/latest/job-scheduling.html#scheduling-within-an-application" target="_blank">fair scheduling pool</A>.&nbsp;</P><P></P><P>To enable multiple streaming queries to execute jobs concurrently and to share the cluster efficiently,&nbsp;you can set the <A href="https://docs.databricks.com/spark/latest/structured-streaming/production.html#configure-apache-spark-scheduler-pools-for-efficiency" alt="https://docs.databricks.com/spark/latest/structured-streaming/production.html#configure-apache-spark-scheduler-pools-for-efficiency" target="_blank">queries to execute in separate scheduler pools</A></P> Fri, 25 Jun 2021 20:38:57 GMT https://community.databricks.com/t5/data-engineering/how-to-run-multiple-spark-streaming-application-on-databricks/m-p/20692#M14001 sajith_appukutt 2021-06-25T20:38:57Z