Databricks

MarcLimotte · 07-21-2015

I have a fairly small, simple DataFrame, month:month.schema org.apache.spark.sql.types.StructType = StructType(StructField(month,DateType,true), StructField(real_month,TimestampType,true), StructField(month_millis,LongType,true))The month Dataframe i...

MarcLimotte · 07-24-2015

It seems to work as I expect now, even without doing the explicit dbutils.fs.rm(...). Must have been some intermittent problem.

MarcLimotte · 07-22-2015

@Vida Ha I just did "Run All" on a clone of the "BroadcastHashJoin" notebook, and it appears to have errors.On %sql ANALYZE TABLE my_small_table COMPUTE STATISTICS noscanwhich appears right after "Configure a BroadcastHashJoin for a small table.", ...

MarcLimotte · 07-21-2015

BTW, I'm on a Spark 1.4 Databricks cluster.

Databricks

User Stats

User Activity

Why do I get 'java.io.IOException: File already exists' for saveAsTable with Overwrite mode?

Re: Why do I get 'java.io.IOException: File already exists' for saveAsTable with Overwrite mode?

Re: My Spark SQL join is very slow - what can I do to speed it up?

Re: Why do I get 'java.io.IOException: File already exists' for saveAsTable with Overwrite mode?