explicit casting seems like the way to go.
First try with one column, to see if that solves your issue.
If so, you can write a function that casts all decimal columns to a certain precision, something like this:
def convert_decimal_precision_scale(df, precision, scale):
# Iterate through each column and convert decimal columns to the specified precision and scale
for column in df.columns:
data_type = df.schema[column].dataType
if isinstance(data_type, DecimalType):
df = df.withColumn(column, col(column).cast(DecimalType(precision, scale)))
return df