Hey @lchicoma , sorry for the delayed response. Thanks for sharing the error and contextโthis looks like a parsing issue in the feature specification rather than a problem with Delta or the runtime versions.
What changed recently
There was an incident during a rollout of the new server-side FeatureSpec generation in September 2024 that caused FeatureSpec creation to fail with errors like โFollowing nodes form a cycleโ for certain configurations, especially when feature/table names had uppercase characters or when features werenโt explicitly listed. This rollout was rolled back and the issue mitigated at the time. As part of that incident, teams observed that explicitly listing features and using all-lowercase names avoided the failure while the fix was being rolled out. Similar โcycleโ errors have been seen when the lookup key is a string and the system misinterprets it during parsing, though this wasnโt a documented limitation of FeatureLookup itself. The error class INVALID_PARAMETER_VALUE youโre seeing is a generic validation error category used across Databricks services when inputs canโt be parsed or validated.
Why you might see โFollowing nodes form a cycle: accountIdโ
A โcycleโ here means the feature graph constructed from your spec references itself. Common causes: * The primary key (for example, accountId) is also included as a feature output or referenced by a FeatureFunction that indirectly produces a column with the same name, creating a self-loop in the graph.
-
The lookup_key_list (or equivalent) was passed in a format that the parser now treats differently, such as a single comma-separated string instead of a list, leading the generator to create a node that points back to itself. During the 2024 ramp-up, differences in how keys/features were inferred triggered cycle errors; explicitly listing features and using lowercase names avoided it.
-
Mixed-case names in tables or features can still occasionally cause server-side spec generation to infer dependencies incorrectly when features arenโt explicitly listed; switching to lowercase names for the spec inputs is a quick test/workaround.
-
If your โfeature tableโ is actually a view, server-side generation wonโt support it and can fail with related parsing/validation errors. Use a physical table for feature lookups.
Practical fixes to try
Try these in order; theyโre safe, low-effort checks that often resolve this exact error:
-
Ensure your keys are not included in the feature outputs. Only non-key columns should be in your feature_names list.
-
Pass lookup keys as a list of strings (not a single comma-separated string or other structure). For example: ```python from databricks.feature_engineering import FeatureEngineeringClient, FeatureLookup
fe = FeatureEngineeringClient()
fl = FeatureLookup( table_name="catalog.schema.my_online_table", lookup_key_list=["accountId", "ruleModelVersionDesc"], feature_names=["feature_a", "feature_b", "feature_c"] # exclude keys )
spec = fe.create_feature_spec( name="catalog.schema.my_feature_spec", features=[fl] # only lookups/functions; no key columns as outputs ) ```
-
Explicitly list the features instead of relying on auto-inference. That avoids the paths where server-side generation needs to guess and can create cycles.
-
Temporarily standardize names to all lowercase in the spec (keys, table identifier, feature names) and re-try. This has historically avoided the parser issue seen during server-side generation ramp-up.
-
Double-check that none of your FeatureFunctions return a column named accountId or ruleModelVersionDesc, and that no transformation reassigns a key column name to a feature output.
-
If your feature source is a view, switch to a physical table (Delta) for the lookup.
Why runtime version doesnโt matter here
This failure occurs during feature spec parsing/validation on the control plane (and server-side generation), not during cluster execution. Changing the Databricks Runtime ML version typically wonโt affect this specific error.
If it still fails If the above doesnโt resolve it:
- Try creating a minimal spec with a single FeatureLookup, the two lookup keys as a list, and one non-key feature; if that works, gradually add features/functions to isolate the component that introduces the cycle.
- If the cycle persists even in a minimal spec, it may be a workspace-specific server-side generation edge case. Reference incident ES-1260877 when you escalate; it documents a prior regression in server-side generation that produced similar โcycleโ errors and the mitigations used.
Cheers, Louis.
