clearly the conversion is not what you expect.

What I mean is: can you check the schema of the dataframe (pyspark dataframe) and see what column types it has.

Because depending on this pandas will cast them or put them into object type.