we have enabled compute schema in system tables, not able to see cluster and node_type tables

[karthik_p]

we have enabled system tables, under system tables we are able to see compute schema, under that we are not able to see cluster and node_type in azure databricks. do we have any limitations for above tables 

[Kaniz_Fatma]

Certainly, karthik_p! Let’s address your query regarding the cluster and node_type tables in Azure Databricks.

Cluster Tables:

• Cluster-related information is essential for managing and monitoring your Databricks clusters. However, there are some considerations to keep in mind:
  • Single Node Clusters: These clusters are intended for jobs with small data amounts or non-distributed workloads (such as single-node machine learning libraries). They are not designed for sharing and cannot be scaled to 0 workers. Additionally, GPU scheduling is not enabled on Single Node clusters1.
  • Multi Node Clusters: For larger jobs with distributed workloads, Multi Node clusters are recommended. They can be shared among multiple users and provide better resource utilization. When sharing clusters, it’s crucial to avoid resource conflicts, and Multi Node clusters help achiev...2.

Node_Type Tables:

• As of now, there are no specific system tables named node_type in Azure Databricks. If you’re unable to find such a table, it’s likely because it doesn’t exist in the system tables.
• Keep in mind that system tables may vary based on the Databricks version, and some features might be in private preview or not yet available. If you have specific requirements related to node types, I recommend reaching out to your Databricks....

Resource Limits:

• It’s essential to be aware of resource limits when working with Azure Databricks. These limits ensure efficient resource utilization and prevent misuse.
• For additional information on resource limits (including clusters), refer to the Azure Databricks Limits documentation.

Remember that your configuration decisions involve a tradeoff between cost and performance. 

If you have further questions or need additional assistance, feel free to ask! 🚀