1) Use sc.parallelize to create the table.

2) Register just a temporary table.

3) You can keep adding insert statements into this table. Note that Spark SQL supports inserting from other tables. So again, you might need to create temporary tables to insert into the first temporary table.

This table should not write out to disk until you run a "saveAsTable" or other type of command.

-Vida