topic Re: Error on Workflow in Get Started Discussions

Error on Workflow

JohnSmith2 — Fri, 22 Sep 2023 08:06:35 GMT

Hi , I have some mysteries situation here

My workflow (job) ran and got an error -> [INVALID_IDENTIFIER] The identifier transactions-catalog is invalid. Please, consider quoting it with back-quotes as `transactions-catalog`.(line 1, pos 12) == SQL == transactions-catalog.transactions-schema.transactions_bronze_table_eu

But when I ran it with manually it was success.

Is anyone have any idea about this

Coding

%python

silver_df = spark.sql(f"""

SELECT

column

from table_changes("`transactions-catalog`.`transactions-schema`.table" , {start_version} , {end_version})

""")

***I conceal a column and a table name.

Re: Error on Workflow

-werners- — Fri, 22 Sep 2023 08:16:00 GMT

that is weird indeed.
Are you sure the manual test you did has the same code as the one in the job (perhaps different REPOS branch?)?

Re: Error on Workflow

JohnSmith2 — Fri, 22 Sep 2023 09:05:14 GMT

I am sure that I use a same code in both.

Re: Error on Workflow

-werners- — Fri, 22 Sep 2023 11:48:37 GMT

Jobs are just notebooks executed in background, so if the notebook is the same between interactive (manual) and job run, there should be no difference.
So I don't see what is wrong. Is the job using DLT perhaps?

Re: Error on Workflow

JohnSmith2 — Sun, 24 Sep 2023 12:13:46 GMT

Ok, Sorry for this problem, I just found the different. It's a version of runtime that on my manual use 12.2 but on my job use 13.3. Thank you for your advice to check the different.