You can disable the serverless compute featuure from your account console :https://docs.databricks.com/aws/en/admin/workspace-settings/serverless#enable-serverless-compute

I have heard that for some ,If this option is not available , it means it is auto enabled in your workspace.
So you should reach out to databricks support team or your account executive to get it back so that you can disable.

Also you can add serverless budget policies and keep is as tight as possible to restrict users. 

- https://docs.databricks.com/aws/en/admin/usage/budget-policies