cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

Databricks-Connect and Change Data Feed query error

Ian
New Contributor III

I have installed Databricks-Connect (9.1 LTS). I am able to send queries to the cluster. However, when the query includes a call to the 'table_changes' function that is a part of Change Data Feed, I get the following error:

AnalysisException("could not resolve `table_changes` to a table-valued function; line 1 

The query runs when I run it in a notebook against the same cluster. The query looks similar to the following:

select distinct SomeFieldID from table_changes('SomeTable','2021-12-31 9:53:02') where _change_type in ('update_postimage')

1 ACCEPTED SOLUTION

Accepted Solutions

Hi @Ian Coreraโ€‹ ,

Please check the limitations from DBconnect. https://docs.databricks.com/dev-tools/databricks-connect.html#limitations It seems like the feature work fine on the notebook but not on DBconnect.

View solution in original post

4 REPLIES 4

Anonymous
Not applicable

Hello @Ian Coreraโ€‹ - My name is Piper, and I'm one of the moderators for Databricks. Let's see what your fellow members have to say before we circle back around to this. ๐Ÿ™‚

Ian
New Contributor III

Hi @Kaniz Fatmaโ€‹ , the table_changes function is an internal Databricks function used in Change Data Feed (CDF).

Please refer to the article below. It discusses the table_changes function.

https://docs.databricks.com/delta/delta-change-data-feed.html

Ian
New Contributor III

Hi @Kaniz Fatmaโ€‹  thank you for your reply. I do have the tables setup for Change Data Feed. I am able to run the query with 'table_changes' in a notebook. However, the same query does not run in Databricks-Connect (9.1 LTS).

In Databricks-Connect I get the error below:

AnalysisException("could not resolve `table_changes` to a table-valued function; line 1 

Again, the query runs fine in a notebook in a Databricks workspace.

Hi @Ian Coreraโ€‹ ,

Please check the limitations from DBconnect. https://docs.databricks.com/dev-tools/databricks-connect.html#limitations It seems like the feature work fine on the notebook but not on DBconnect.

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโ€™t want to miss the chance to attend and share knowledge.

If there isnโ€™t a group near you, start one and help create a community that brings people together.

Request a New Group