Serverless giving inconsistent results in Oracle UCM SOAP call

[maddan80]

Hello ,

We have implemented Data pipeline to ingest data from Oracle UCM using SOAP API, This was working fine with Job and all Purpose clusters. Recently we wanted to use Serverless to take advantage of the server startup time. In this case we were not receiving the correct response when executed thru serverless. Especially the part when service using 

query_text = f"dDocTitle <starts> `{manifest_file_prefix}`" for the

 <ns1:Service IdcService="GET_SEARCH_RESULTS">

which needs to return the documents associated to the manifest. In Job clusters it returns, but in Serverless it does not return any records eventhough it return status code as 200.  We tried other SOAP calls including BICC in Oracle UCM  which does not require the query text which is working fine for both Serverless and Job clusters. So there is no issue in reaching the end point in terms of Serverless and Job compute. Suspecting how the query text parameter is parsed in Serverless and Job compute. 

Tried other options 

query_text = "dDocTitle <contains> 'manifestid-'" it gave 500 error for both computes.

Highly appreciate any help in this.

[mmayorga]

Hi @maddan80 

Thank you for reaching out with your question and providing the context about your use case.

Per your comments, having a 200 Status Code in Serverless is a good initial indicator that the request is reaching the Oracle UCM server. Brainstorming... the problem could be in how the response is handled in a different environment (Serverless)

There are Serverless environment versions with different configurations of Python, Scala, and the JDK. Use a version that matches the configuration used in your all-purpose and job clusters.

BTW, in Lakeflow Jobs, you can configure an environment for each of your job tasks where you can select a specific Version and dependencies

Other considerations: 

• Check for any Serverless limitations 
• Do you have a unit test that can validate a simple call (such as a health check) before requesting data? This could help determine whether the issue is general or specific to your search, such as Serverless being unable to reach the Oracle UCM host.
• Is the "manifest_file_prefix" variable checked for encoding as well? This could potentially mess with the query_text variable and/or affect the results.
• Even with a 200 Status, look further in the SOAP body's response and explicitly check for expected nodes.

I hope this helps! 
Thank you