That's actually not a Databricks problem per se but relates to the HTML <video> tag. You could use your browser's developer tool, inspect the video element, and find the video source. Then you could open that URL (hosted by some CDN service) in a new tab where you can enable fullscreen mode.
On Databrick's side, they could of course implement a proper JavaScript function to enable fullscreen mode.