romy
Databricks Employee
Databricks Employee

Hi, the show() method prints only the top 20 rows by default: DataFrame.show(n: int = 20truncate: Union[bool, int] = Truevertical: bool = False) (cf https://spark.apache.org/docs/latest/api/python/reference/pyspark.sql/api/pyspark.sql.DataFrame.show...)

You can either use show() with a bigger n parameter, or use the Databricks display() command to print the dataframe in a tabular format:

df = spark.read.format("CSV").option("inferSchema", "true").option("header", "true").option("delimiter", ",").load(file_location)

display(df)

https://www.databricks.com/spark/getting-started-with-apache-spark/dataframes#view-the-dataframe