cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Get Started Discussions
Start your journey with Databricks by joining discussions on getting started guides, tutorials, and introductory topics. Connect with beginners and experts alike to kickstart your Databricks experience.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

Snowflake Data Formatting Issue

rohithmalla
New Contributor

I'm loading snowflake data to delta tables in databricks, few columns in snowflake data have datatype as Number (20,7) after loading to delta table it is taking as decimal (20,7), for example, if the value is 0.0000000 in snowflake then it is showing as 0E-7 in delta table, but i need it as 0.0000000 in delta table, I can't perform formatting function because there are many columns with data type as Number and with different decimal ranges, Please suggest how can I load it as 0.0000000

1 REPLY 1

-werners-
Esteemed Contributor III

explicit casting seems like the way to go.
First try with one column, to see if that solves your issue.
If so, you can write a function that casts all decimal columns to a certain precision, something like this:

def convert_decimal_precision_scale(df, precision, scale):
    # Iterate through each column and convert decimal columns to the specified precision and scale
    for column in df.columns:
        data_type = df.schema[column].dataType
        if isinstance(data_type, DecimalType):
            df = df.withColumn(column, col(column).cast(DecimalType(precision, scale)))
    return df

 

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโ€™t want to miss the chance to attend and share knowledge.

If there isnโ€™t a group near you, start one and help create a community that brings people together.

Request a New Group