Monitor all Streaming jobs to make sure they are in RUNNING status.

[sensanjoy]

Hi Experts,

Is there any way that we can monitor all our Streaming jobs in workspace to make sure they are in "RUNNING" status?

I could see there is one option to create a batch job that runs frequently and check the status(through REST API) of all streaming jobs and trigger it when its not running.

Do we have any other solution from job level configuration or setup an alert for job not running!!

Thanks. 

[Kaniz]

Hi @sensanjoy, - You can monitor streaming jobs in the workspace through:

 1. **Job Run Dashboard**: Displays information about all running jobs.
    - Components: Job ID, Run Page, Run Name, Start Time, Created By.
    - Divided into two sections: Job Runs (scheduled jobs) and Run Submits (jobs invoked via API).
    - Can be configured to run every minute.

 2. **View Recent Job Runs**: This lists currently running and recently completed runs for all accessible jobs.
    - Includes a table of job runs from the last 67 days.
    - Details on failed, skipped, and successful job runs.
    - View:
       

python
        # Click Workflows in the sidebar.
        # Click the Job runs tab to display the Job runs list.
        

 3. **Configure Jobs to Restart Streaming Queries on Failure**: Create a job with the notebook or JAR containing streaming queries and configure it to always use a new cluster and retry on failure.

[sensanjoy]

Thanks@Kaniz .

If we plan in programmatic way it might required less manual intervention. what do you suggest and Rest API can be used in this use case.