VZLA
Databricks Employee
Databricks Employee

@nikhil_kumawat can you provide more details to reproduce this and better help you? e.g.: sample data set, dbr version, reproducer code, etc.

I'm having this sample data:

csv_content = """column1,column2,litre_val,another_decimal_column
1,TypeA,60211.952,12.3459
2,TypeB,59164.608,45.6789
3,TypeC,12345.678,78.9012
"""

Which I'm then storing as csv file in my dbfs temp location. Then I'm reading it back without a schema, but simple inference:

# Reading the CSV file without using any schema
df = spark.read.format("csv").option("header", "true").load("/some/path/to/test_data.csv")

And when displaying it using:

df.show(truncate=False)
df.printSchema()

I'm seeing the results as:

+-------+-------+---------+----------------------+
|column1|column2|litre_val|another_decimal_column|
+-------+-------+---------+----------------------+
|1      |TypeA  |60211.952|12.3459               |
|2      |TypeB  |59164.608|45.6789               |
|3      |TypeC  |12345.678|78.9012               |
+-------+-------+---------+----------------------+

root
 |-- column1: string (nullable = true)
 |-- column2: string (nullable = true)
 |-- litre_val: string (nullable = true)
 |-- another_decimal_column: string (nullable = true)

Using display(), does not alter the results:

Screenshot 2025-01-03 at 11.20.27.png