2-For any background process, will there be any number of cores and its memory allocated?

Yes, background processes in Databricks also utilize resources, but their impact on core and memory allocation depends on the workload and the specific processes running. Some common background processes and their resource usage include:

• Driver and Executor Management: The Databricks environment handles the management of the driver and executor processes which run in the background. These processes are allocated cores and memory as needed based on the workload and cluster configuration.
• Job Scheduling and Resource Management: Databricks handles job scheduling and resource allocation, which involves some background processes to ensure efficient resource utilization.
• Monitoring and Logging: Background processes for monitoring cluster performance and logging system metrics use a portion of the available resources.