I have a table in Databricks that stores job information, including fields such as job_name, job_id, frequency, scheduled_time, and last_run_time.

I want to run a query every 10 minutes that checks this table and triggers a job if the scheduled_time is less than or equal to the current time.

Some jobs have multiple frequencies, for example, the same job might run daily and monthly. In such cases, I want the lower-frequency job (e.g., monthly) to take precedence, meaning only the monthly job should trigger and the higher-frequency job (daily) should be skipped when both are due.

What is the best way to implement this scheduling and job-triggering logic in Databricks?