Hi all,

I'm encountering an INTERNAL_ERROR issue when refreshing a Databricks Lakehouse Monitoring job. Here's the full error message:

`ProfilingError: INTERNAL_ERROR. Please contact the Databricks team for further assistance and include the refresh id.`

I'm using Lakehouse Monitoring via YAML configuration with a granularity of 1 day. I'm also using custom metrics like:

```custom_metrics:
- name: "subscription_conversion_rate"
definition: "CASE WHEN column_1 > 0 THEN column_1 / column_2 ELSE 0 END"
type: "CUSTOM_METRIC_TYPE_AGGREGATE"
output_data_type: "DOUBLE"
input_columns: ['column_1', 'column_2']
- name: "total_count_per_brand"
definition: "1"
type: "CUSTOM_METRIC_TYPE_AGGREGATE"
output_data_type: "INT"
input_columns: ['brandcode']```

Has anyone experienced this? Any insights or debugging tips would be appreciated!

Thanks in advance!