I think there is some kind of problem with networking/permissions to the storage account created in managed resource group by Databricks. By default, when you run a notebook interactively by clicking Run in the notebook:

• If the results are small, they are stored in the Azure Databricks control plane, along with the notebook’s command contents and metadata.
• Larger results are stored in the workspace storage account in your Azure subscription. Azure Databricks automatically creates the workspace storage account. Azure Databricks uses this storage area for workspace system data and your workspace’s DBFS root. Notebook results are stored in workspace system data storage, which is not accessible by users. 

So in your case, when you limit the result set then it works becasue small results are stored in Azure Databricks control plane.
But when you try to display whole datframe without limiting it, databricks will try to save result in the workspace storage account. Look at the cluster logs and see if there is some errors related to the root storage account.
Maybe you have some firewall that prevents Databricks to connect to storage account.