Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
01-03-2025 02:22 AM
@nikhil_kumawat can you provide more details to reproduce this and better help you? e.g.: sample data set, dbr version, reproducer code, etc.
I'm having this sample data:
csv_content = """column1,column2,litre_val,another_decimal_column
1,TypeA,60211.952,12.3459
2,TypeB,59164.608,45.6789
3,TypeC,12345.678,78.9012
"""
Which I'm then storing as csv file in my dbfs temp location. Then I'm reading it back without a schema, but simple inference:
# Reading the CSV file without using any schema
df = spark.read.format("csv").option("header", "true").load("/some/path/to/test_data.csv")
And when displaying it using:
df.show(truncate=False)
df.printSchema()
I'm seeing the results as:
+-------+-------+---------+----------------------+
|column1|column2|litre_val|another_decimal_column|
+-------+-------+---------+----------------------+
|1 |TypeA |60211.952|12.3459 |
|2 |TypeB |59164.608|45.6789 |
|3 |TypeC |12345.678|78.9012 |
+-------+-------+---------+----------------------+
root
|-- column1: string (nullable = true)
|-- column2: string (nullable = true)
|-- litre_val: string (nullable = true)
|-- another_decimal_column: string (nullable = true)
Using display(), does not alter the results: