[INTERNAL_ERROR] Cannot refresh quality dashboard

[Aneruth]

Hi all,

I'm encountering an INTERNAL_ERROR issue when refreshing a Databricks Lakehouse Monitoring job. Here's the full error message:

`ProfilingError: INTERNAL_ERROR. Please contact the Databricks team for further assistance and include the refresh id.`

I'm using Lakehouse Monitoring via YAML configuration with a granularity of 1 day. I'm also using custom metrics like:

```custom_metrics:
- name: "subscription_conversion_rate"
definition: "CASE WHEN column_1 > 0 THEN column_1 / column_2 ELSE 0 END"
type: "CUSTOM_METRIC_TYPE_AGGREGATE"
output_data_type: "DOUBLE"
input_columns: ['column_1', 'column_2']
- name: "total_count_per_brand"
definition: "1"
type: "CUSTOM_METRIC_TYPE_AGGREGATE"
output_data_type: "INT"
input_columns: ['brandcode']```

Has anyone experienced this? Any insights or debugging tips would be appreciated!

Thanks in advance!

[Aneruth]

Thank you! I'll modify my query based on your explanation. Currently, I'm manually parsing the custom metrics output data types, which works but isn't ideal. I'll implement proper data type formatting through asset bundles to ensure the UI receives correctly typed metrics data.