HI @Fred Foucart​ ,

The above code looks good to me. Can you try with below code as well.

spark.read\

  .format("jdbc") \

  .option("url", f"jdbc:postgresql://{host}/{database}") \

  .option("driver", "org.postgresql.Driver") \

  .option("user", username) \

  .option("password", password) \

  .option("dbtable", <TableName>) \

  .option("fetchsize", 5000) \

  .load()

In case the table is huge, you can try with parallel reads.

#Param 

table_name = "<Your Table Name>"

partitionColumn = "<Primary Key Numeric Column>"

lowerBound = 1

upperBound = 10000 <Total Row Count> #Our table contains over a Billion rows!!!

fetchsize = 1000

num_partitions = 20 # Do some math on how many slices the data should be partitioned 

          and read (Total Records / Partitions) i.e. 10000/20 = 500 rows per thread

#Read 

source_df = spark.read\

  .format("jdbc") \

  .option("url", f"jdbc:postgresql://{host}/{database}") \

  .option("driver", "org.postgresql.Driver") \

  .option("user", username) \

  .option("password", password) \

  .option("dbtable", source_table) \

  .option("partitionColumn", partitionColumn) \

  .option("lowerBound", lowerBound) \

  .option("upperBound", upperBound) \

  .option("numPartitions", partitions) \

  .option("fetchsize", fetchsize) \

  .load()