topic Re: View Failing due to filed not recognise in Data Engineering

View Failing due to filed not recognise

Anuj_Diggibyte — Mon, 04 Aug 2025 06:40:30 GMT

Hi,

I am facing problem where my existing view has stopped working due to unrecognised filed which is a alias field .

I am using the same definition and spark configuration.

DBR : 14.3 LTS, Spark 3.5.0

Any one faced similar problem recently ?

Re: View Failing due to filed not recognise

Khaja_Zaffer — Mon, 04 Aug 2025 06:49:32 GMT

Hello @Anuj_Diggibyte

good day!

Is select * working as expected? just remove those filters and confirm once?

Re: View Failing due to filed not recognise

Khaja_Zaffer — Mon, 04 Aug 2025 06:54:12 GMT

Can you try this?

select *
from team_finance_global.vw_fact_pnl_exp001
where "Id Date" = "20250807"

Try with double quoats

-- This assumes 'id_date' is the correct underlying column name
select *
from team_finance_global.vw_fact_pnl_exp001
where id_date = '20250807'

id_date is the original column name, but the alias is 'Id Date'. Using the alias with the appropriate quoting is the intended way to query the view.

Re: View Failing due to filed not recognise

Anuj_Diggibyte — Mon, 04 Aug 2025 07:12:25 GMT

Re: View Failing due to filed not recognise

Khaja_Zaffer — Mon, 04 Aug 2025 07:38:05 GMT

Okay, when you run select * where you able to see any data? just avoid filters for now.

Re: View Failing due to filed not recognise

Anuj_Diggibyte — Mon, 04 Aug 2025 08:04:21 GMT

it is working without filter

Re: View Failing due to filed not recognise

Khaja_Zaffer — Mon, 04 Aug 2025 08:55:48 GMT

Hello @Anuj_Diggibyte

it seems like the given date is not present.

May be you can confirm with the above syntax and check filter with date 20241031, may be 20250131 is not present?

If thats not the expectation, I expect others to jump in for the solutions thank you have a great day!

Re: View Failing due to filed not recognise

Anuj_Diggibyte — Mon, 04 Aug 2025 13:55:07 GMT

HI @Khaja_Zaffer ,

The Date is present in data .

The problem i see here :

The renamed column in select clause is not identifying with single quotes " ` " when we are passing with double quotes then it is considering the column but filtering is not working properly i am getting empty df.

I am suspecting some setting changed in cluster which is responsible for quotes behaviour .

Re: View Failing due to filed not recognise

Khaja_Zaffer — Sat, 09 Aug 2025 08:50:30 GMT

Hello @Anuj_Diggibyte

is it resolved?

Re: View Failing due to filed not recognise

Anuj_Diggibyte — Mon, 11 Aug 2025 05:06:57 GMT

HI @Khaja_Zaffer

yes it was actaully issue with privacera version .

after invastaigation with privacera team they suggested to upgrade the privacera version to 9.0.35.1.