How to export full result Databricks Azure

[User16826994223]

what is the best way to see all the data , I see display shows up to 100000 data only . any way in which I can see all the data or do I need to download or export it in different file

[User16826994223]

Yes, databricks display only a limited dataframe. It allows you to download the data like a csv, . You can save the dataframe as a table in the databricks database with this:

predictions.select("salry", "dept").write.saveAsTable("depsalry")

Then you can load it with:

predictions = spark.table('predictions')