Float Value change when Load with spark? Full Path?

[Hola1801]

Hello,

I have created my table in Databricks, at this point everything is perfect i got the same value than in my CSV.

for my column "Exposure" I have :

0         0,00
1         0,00
2         0,00
3         0,00
4         0,00
...

But when I load my file with spark, in the column exposure I have something different:

0        14 032,24
1        14 032,24
2         8 061,94
3         8 061,94
4        15 506,37

I use this code to load by table:

import pandas as pd
 
df = spark.table("imos_prior").toPandas()
df['Exposure']

Do you know how could specify the full path of my table ?

My table is located by default in default/imos_prior

Or do you have any idea why the value could have change ? I hope is just a question of different file that is used by sparks.

Thanks you

[Anonymous]

Hello @Anis Ben Salem​ - My name is Piper and I'm a moderator for Databricks. Welcome and thank you for posting your question. Let's give it a bit longer for other members to respond. If we don't hear anything, we'll circle back around.

[-werners- · Accepted solution]

do you happen to know the path where the actual data resides?

Tables in databricks are not the actual data but like a view on top of the data (parquet, csv etc)

[jose_gonzalez]

Hi @Anis Ben Salem​ ,

How do you read your CSV file? do you use Pandas or Pyspark APIs? also, how do you created your table?

could you share more details on the code you are trying to run?