Got it.
My background is not SWE, I have always been a 'data guy', but I definitely appreciate a proper dev workflow (ci/cd, git integration, tests).
When we started using databricks like 7 or 8 years ago, we went for notebooks as this got us up to speed fast and we could deliver very fast.
I have nothing against notebooks, the code is executed as any other code.
BUT, as you also mention, they do not promote good code practice (modularity mainly) and testing.
That is why I decided to drastically change our way of working, which was alsmost exactly as you are doing now.
It became clear that when external consultants joined our team, we had to step up and become way stricter in the way code was promoted to prod.
So I looked into asset bundles and ci/cd pipelines.  And that is what we start using now.
We still use notebooks, but only like a 'main' program that executes/calls functions/methods and writes the data.
All the rest sits in .py files and config files.
We use pytest for unit tests.
Classes that are used in many places are packaged into a wheel (by asset bundles!)
On each commit to git, unit tests run (locally!), after each successful PR merge, we can run integration tests and end to end tests (on a databricks cluster).
The latter 2 are not yet fully in practice due to a lack of time at the moment but we definitely will go there.

So, do not despair using notebooks. You can still modularize. 

But be aware that if you import .py files, databricks looks only into the CWD, so you will either have to append to the python path or package them in wheels.
For testing, databricks made some guidelines on how to test with notebooks:
https://docs.databricks.com/aws/en/notebooks/test-notebooks
Franky I don't like it.  I don't want to run unit tests on a databricks cluster (not necessary). But it might help you.

A reason why you might look into DAB is to programmatically create jobs.

Then there is also serverless compute since a short while. That can be interesting for short running tests.
I do not use it because there is no way to set environment vars on the serverless cluster (yet).

So about those 'missing' files: I never had any issue with that.  We put notebooks etc in the workspace, data in UC.
Works without a problem.
DBFS however is indeed not recommended anymore.

That being said:
I am far from happy how Databricks is making decent code deployment harder and harder.  It should be simple and solid so an engineer can focus on writing code.
Heck, we even have to stop using scala because it is clear that there is no future for that language in databricks.  New features are released on python/sql first, and maybe scala some time later.
I am still a fan of the platform though, I just wish they gave engineers some attention.

So hopefully this helps a bit.