Since this week it went away automatically. I don't know why, I think Databricks changed something under the hood. The Databricks platform engineering team at my company couldn't figure out the cause, and also our resident Databricks solution architect could reproduce the issue on his own (Databricks internal) workspace and didn't know how it could be solved.

So I'm happy the problem magically disappeared for me, but unfortunately can offer you little advice beyond "wait and hope it will disappear" =(