Feature Store UI lagging

[Kjetil]

The feature store UI is lagging (Azure Databricks). It is extremely slow to update - takes several days from creating a feature table before it appears in the UI. Anyone share the same experience?

[Kaniz]

Hi @Kjetil, I’m sorry to hear that you’re experiencing lag with the Azure Databricks feature store UI.

The Databricks feature store is a centralized repository that enables data scientists to find and share features. It ensures that the same code used to compute the feature values is used for model training and infe.... However, developing features can be complex and time-consuming, which can lead to delays.

Here are a few general steps you can take to diagnose performance issues:

1. Use the Jobs Timeline to identify major issues.
2. Look at the longest stage.
3. Look for skew or spill.
4. Determine if the longest stage is I/O bound.
5. Look for other causes of slow stage runtime.

[Kjetil]

Thanks for replying.

Just to be clear this is a problem with the Databricks UI. There is absolutely no issue with the job itself, everything runs smoothly and the feature tables are registered. This is purely an issue with the Databricks Feature Store UI (the tab at the left called Feature Store). It takes ages for new tables to appear after the tables have been registered (I have doble checked that they are indeed registered as feature tables using the FeatureEngineeringClient. Moreover, there is not only an issue with new feature tables not appearing - all changes to feature tables (like changing ownership of the table) takes a very long time before appearing in the Feature Store UI. So it is a frontend issue, and I have no idea how to solve it, but it would be nice to be able to use the UI for browsing features.

rgs Kjetil