topic Re: Delete feature tables through the Python API in Machine Learning

Delete feature tables through the Python API

GabrieleMuciacc — Tue, 08 Mar 2022 14:02:34 GMT

The documentation explains how to delete feature tables through the UI. Is it possible to do the same using the Python FeatureStoreClient? I cannot find anything in the docs: https://docs.databricks.com/_static/documents/feature-store-python-api-reference-0-3-7.pdf

Partially related: is it possible to manage control access through Python APIs too?

Use case: we use ephemeral dev environments for development and we have automated deletion of resources when the environment is torn down. Now we are considering using the feature store, but we don't know how to automate deletion.

Re: Delete feature tables through the Python API

Hubert-Dudek — Tue, 08 Mar 2022 15:17:31 GMT

You can try to delete that table through metastore:

%sql DROP TABLE IF EXISTS <feature_table_name>

Re: Delete feature tables through the Python API

GabrieleMuciacc — Tue, 08 Mar 2022 17:05:10 GMT

Thanks @Hubert Dudek for the answer. However, this only deletes the underlying Delta table, not the feature table in the store: you end up in an inconsistent state where you cannot write/read and you cannot re-create the table.

Re: Delete feature tables through the Python API

Hubert-Dudek — Tue, 08 Mar 2022 17:39:12 GMT

@Kaniz Fatma @Piper Wilson maybe someone from Databricks team could check is there any API command or API endpoint or SQL command or to delete feature tables? I couldn't find any and workaround which I proposed create inconsistent state.

Re: Delete feature tables through the Python API

GabrieleMuciacc — Wed, 09 Mar 2022 10:14:14 GMT

Hi @Kaniz Fatma , unfortunately this is the same link I included in my question and it does not help. I was asking about doing it programmatically.

Re: Delete feature tables through the Python API

Hubert-Dudek — Wed, 09 Mar 2022 10:41:07 GMT

@Kaniz Fatma no this is through UI, request is how to do it programatically

Re: Delete feature tables through the Python API

Atanu — Fri, 13 May 2022 06:08:32 GMT

I am not 100% familar with this but does this relates to feature store. if yes, API works it seems https://docs.databricks.com/applications/machine-learning/feature-store/python-api.html. Please let me know if this help @Gabriele Muciaccia Thanks.

Re: Delete feature tables through the Python API

GabrieleMuciacc — Fri, 13 May 2022 07:06:47 GMT

Thanks, I see it has been added as an experimental method in version 0.4.1 (Databricks Runtime 10.5 for ML).

There is still no way to control access though, hopefully that will be added too.

Re: Delete feature tables through the Python API

GabrieleMuciacc — Fri, 13 May 2022 08:44:41 GMT

I have selected as best, but @Atanu Sarkar could you please edit your answer to include feature store and Databricks Runtime versions? Thanks!

Re: Delete feature tables through the Python API

SonghuaZhang — Wed, 03 Aug 2022 08:52:47 GMT

Setting permission to feature table programmatically will be much helpful. Another API I think that will be useful is the search function. and even better, if we can have a databricks (restful) API endpoint for feature store.

Re: Delete feature tables through the Python API

User16788316720 — Tue, 20 Dec 2022 09:40:32 GMT

from databricks import feature_store
fs = feature_store.FeatureStoreClient() 
fs.drop_table(FEATURE_TABLE_NAME)

As of Databricks Runtime 10.5 for ML. Docs