Merge error in streaming case

Remit
New Contributor III

I have a streaming case, where i stream from 2 sources: source1 and source2. I write to seperate streams to pick the data up from the landing area (step1). then i write 2 extra streams to apply some tranformations in order to give them the same schema and have the 2 streams write them to the same target (let's call it union_stream) (step 2). So far everything works fine. Then i write another stream to apply some tranformations to union_stream and write it to union_stream2 (step 3).  In this step all the data from source 1 flows perfectly through the stream but the data from source 2 doesn't. When i filter on source_name = source2, i get the error: Keeping the source of the MERGE statement materialized has failed repeatedly. Even when i execute step 3 without streaming i get the same error. I have also tried to not perform any transformations in step 3 and just do a MERGE into a table with the exact same schema, this led to the same error.

Any help would be appreciated. Thx in advance

Remit
New Contributor III

Solved the problem by changing the cluster settings. The whole thing works when disabling Photon Acceleration...

View solution in original post