To sum it up. Official documentation is saying following: 

https://learn.microsoft.com/en-us/azure/databricks/notebooks/use-databricks-assistant#enable-or-disa...

For an account: Disable or enable Databricks Assistant features

Azure AI-powered Databricks Assistant features are enabled by default. However, account admins can disable Azure AI-powered Assistant features for all workspaces within an account. Account admins can also enforce enabling Databricks Assistant for the entire account.

If your account admin enables Assistant, it is enabled by default in workspaces. Unless the account admin chose to enforce the setting, workspace admins can disable it for their workspaces.

So, currently there is no way to disable it only for specific user.