@bkr, you can reference the file name using dbutils and then pass this to the move command. Here's an example for this in Scala:
val fileNm = dbutils.fs.ls("/usr/krishna/sample").map(_.name).filter(r => r.startsWith("test"))(0)
val fileLoc = "dbfs:/...
@sdaza, this is a compatibility issue. As a workaround, you can make a manual update to address via the code below.
# TIMESTAMP IMPORT MUST BE CHANGED WHEN USING PANDAS 23.0+
#from pandas.lib import Timestamp
# CHANGE TO THE FOLLOWING
from pandas._...
@sdaza,
You can go from a Spark Data frame to pandas and visualize with matplotlib or from pandas to Spark data frame (separate block) using the methods below. The syntax for the pandas plot is very similar to display(<data frame>) once the plot is...
Hi @sdaza,
You can use the display command to display objects such as a matplotlib figure or Spark data frames, but not a pandas data frame. Below is code to do this using matplotlib. Within Databricks, you can also import your own visualization lib...
Hi @sdaza,
The display command can be used to visualize Spark data frames or image objects but not a pandas data frame. If you'd like to visualize your pandas data, I recommend using matplotlib to prep the data into a figure. Code below showing how ...
