Question: Issue with Overwatch Deployment on Databricks (on AWS) - Missing Tables in Gold Schema

jiteshraut20
New Contributor III

Hi all,

I'm working on setting up Overwatch in our Databricks workspace to monitor resources, and I've encountered an issue during overwatch deployment. I am able to deploy overwatch, but the validation for the `Gold_jobRunCostPotentialFact` module failed with the following error:

> "Unsupported component type interface scala.collection.Seq in arrays."

Due to this error, a table is missing in the gold schema: `overwatch_global.jobRunCostPotentialFact`. this tables is essential for our reporting needs.

What I've tried so far:
- I attempted using different versions of the Overwatch libraries installed on the compute cluster, but the same error persists.
- I deployed each layer of the Medallion architecture for Overwatch separately. While all deployments were successful, the mentioned error still appeared in the logs, and the table remains missing from the consumer database.

I am using Scala version 2.12, Spark version 3.5.0 and DBR 15.3 ML

Has anyone else encountered this issue or have suggestions on how to resolve it? Any help would be greatly appreciated!

Thanks in advance!

Jitesh Raut