Yes, databricks display only a limited dataframe. It allows you to download the data like a csv, . You can save the dataframe as a table in the databricks database with this:
predictions.select("salry", "dept").write.saveAsTable("depsalry")
Then you can load it with:
predictions = spark.table('predictions')
