Databricks bundle
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
10-17-2024 04:13 AM
Hey,
I am new to Databricks, and I am trying to test the mlops-stack bundle.
Within that bundle there is a feature-engineering workflow and I have a problem to make it run.
The main problem is the following.
the bundle specified the target to be $bundle.target which is in my case would be dev. I have created the dev catalog and within the project schema according to the template.
The issue is that when I run the workflow, the notebook fails at
from databricks.feature_engineering import FeatureEngineeringClient
fe = FeatureEngineeringClient()
# Create the feature table if it does not exist first.
# Note that this is a no-op if a table with the same name and schema already exists.
fe.create_table(
name=output_table_name,
primary_keys=[x.strip() for x in pk_columns.split(",")] + [ts_column], # Include timeseries column in primary_keys
timestamp_keys=[ts_column],
df=features_df,
)
# Write the computed features dataframe.
fe.write_table(
name=output_table_name,
df=features_df,
mode="merge",
)I am getting that:
ValueError: Catalog 'dev' does not exist in the metastore.
And I don't understand why?. If I ran the notebook through my own cluster.
I tried to give all privileges to all users in the workspace, but it did not help.