Hi! Kedro Dev here. You can surely develop Kedro on Databricks, in fact we have a lot of Kedro project running on Databricks.
In the past there has been some friction, mainly because Kedro are project based while Databricks focus a lot on notebook. This has changed since the introduction of dbx, data assets bundle which allows user work in a IDE while sending job to databricks. There is now a new kedro-databricks community plugin which helps you to get started quickly with data assets bundle https://github.com/JenspederM/kedro-databricks.
> I've been seeing companies opt for MLflow 2.0 (recipes) and DLT which essentially does the same thing as Kedro with native support.
Kedro are quite different from these tools, but I agree it does not make too much sense if you are using Recipe and Kedro at the same time. I won't be able to give a fair comparison since I am obviously biased.