cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
cancel
Showing results for 
Search instead for 
Did you mean: 

How can I use display() in a python notebook with pyspark.sql.Row Objects, e.g. after calling the first() operation on a DataFrame?

Anonymous
Not applicable

I'm trying to

display()
the results from calling
first()
on a DataFrame, but
display()
doesn't work with
pyspark.sql.Row
objects. How can I display this result?

2 REPLIES 2

Anonymous
Not applicable

The

display()
function requires a collection as opposed to single item, so any of the following examples will give you a means to displaying the results:

  • `display([df.first()])` # just make it an array
  • display(df.take(1))
    # take w/ 1 is functionally equivalent to first(), but returns a DataFrame
  • display(df.limit(1))
    # I'm not 100% sure this is guaranteed to be ordered, but it's another option to try that returns a DataFrame

dnchari
New Contributor II

Use take()