Hello Louis it's me again 🙂

I come back to you after having explored the potential of Lakehouse monitoring. Somethings are bothering me to be honest especially since I am nt at production level at the moment and I would like to ask you some questions

1) Is  the lakehouse monitoring designed for Production only ideally?
2) In my case I am working on an ML usecase. Inference tables would be the optimal choice but do I need a timestamp column for it? My data is pulled on w WEEKLY or BI-WEEKLY basis from Azure Cosmos DB Graph Database. The nodes are imported and saved in a table in databricks. Then a model would be trained after checking data quality and drifts. If there is a drift I would append the newly pulled data from cosmos db to my baseline table (table use for training). The table does not have a Timestamp column but as mentioned the data is pulled on a WEEKLY or BI-WEEKLY basis from Cosmos DB Graph.

3) Can I specify the metrics to be used for drift detection or are they all included? (I recall that the kstest and chi square test are present but there are also other ways to detect drifts.)

4) Does the Lakehouse Monitoring (Inference Tables) work for a Multi Output classification where we would be haviong more than one target column to classify?

5) In my use case I will be having a lot of data volatility, meaning that new labels will introduced, adjusted and new target columns will be introduced on the long term.