cancel
Turn on suggestions
Showing results for 
Search instead for 
Did you mean: 
Help
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Turn on suggestions
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Start a conversation
DejanSunderic
by • New Contributor III
  • 14091 Views
  • 11 replies
  • 3 kudos

is command stuck?

I created some ETL using DataFrames in python. It used to run 180 sec. But it is not taking ~ 1200 sec. I have been changing it, so it could be something that I introduced, or something in the environment.Part of the process is appending results into...

Data Engineering
Reply
  • 14091 Views
  • 11 replies
  • 3 kudos
View Replies
Latest Reply
Carneiro
New Contributor II
  • 3 kudos

I am having a problem very similar. Since yesterday, without a known reason, some commands that used to run daily are now stuck in a "Running command" state. Commands like: dataframe.show(n=1) dataframe.toPandas() dataframe.description() dataframe.wr...

  • 3 kudos
10 More Replies
IgnacioCastinei
by • New Contributor III
  • 56138 Views
  • 9 replies
  • 5 kudos

Resolved! Download a dbfs:/FileStore File to my Local Machine?

Hi all, I am using saveAsTextFile() to store the results of a Spark job in the folder dbfs:/FileStore/my_result. I can access to the different "part-xxxxx" files using the web browser, but I would like to automate the process of downloading all fil...

Data Engineering
Reply
  • 56138 Views
  • 9 replies
  • 5 kudos
View Replies
Latest Reply
CraigJ
New Contributor II
  • 5 kudos

works well if the file is stored in FileStore. However if it is stored in the mnt folder, you will need something like this:https://community.cloud.databricks.com/dbfs/mnt/blob/<file_name>.csv?o=<your_number_here>Note that this will prompt you for yo...

  • 5 kudos
8 More Replies
Labels
Top