After rereading your question, this is quite simple, when downloading a csv from the notebook there will be a down arrow indicator on the right side of the symbol. All you need to do is click that drop down and click download full results (1,000,000...
Hello,I usually use pyspark to do such action. Some example code of how I write it to DBFS from a pyspark data frame:data_frame.coalesce(1).write.format("com.databricks.spark.csv").save("/FileStore/tables/your_file_name.csv")
If you are curious how ...
I would probably use a window function within pyspark.
Link to Databricks blog: https://databricks.com/blog/2015/07/15/introducing-window-functions-in-spark-sql.html
Another option is to use lag or lead columns to help you capture the data in the re...
