Executing NotebookB from NotebookA with arguments, you would use the following syntax within NotebookA to define the arguments:
%run path/to/NotebookB $VarA="ValueA" $VarB="ValueB"
Within NotebookB, you'd use the following to receive the argument va...
Hover between the cells in the side-to-side middle and you will see a + sign appear. This is how you can insert cells into the top-to-bottom middle of a notebook.
You can also move cells by hovering in the upper left of each cell. A cross-hairs will...
Sure. Use the %run syntax as follows:
%run /PoundInclude
The target notebook does not need to be attached to a cluster. It will get pulled into the caller's context.
At this time, you can't combine Scala and Python notebooks, but you can combine Sca...
You can pass parameters/arguments to your SQL statements by programmatically creating the SQL string using Scala/Python and pass it to sqlContext.sql(string). Here's an example using String formatting in Scala:
val param = 100sqlContext.sql(s"""S...
This all depends on the dataset size and specific use cases, but, in general, we've seen that Parquet partitions of about 1GB are optimal. For smaller datasets, however, this large partition size may limit parallelism as tasks operate on individual p...
