Databricks Community

@C P​ @Javier Javier​ @akash kumar​ @sasikumar sagabala​ @Rajesh Jakkula​ @Pratiksha Salwe​ @akash kumar​ @Arjun Krishna S R​ @Dohar Silalahi​ @SparklePandas​ 

During Shuffle operation Data is moving from memory to disk Why?Please find the detailed answer here if any question please comment and hit like and share if interested in upcoming articles.https://medium.com/@sharikrishna26/during-shuffle-operation-...

@DataBricksHelp232​ @Arjun Krishna S R​ @akash kumar​ 

Difference between “ And ‘ in Spark Dataframe APIYou must tell your compiler that you want to represent a string inside a string using a different symbol for the inner string.Here is an example.“ Name = “HARI” “The above is wrong. Why? Because the in...

Spark Dataframes SchemaSchema inference is not reliable.We have the following problems in schema inference:Automatic inferring of schema is often incorrectInferring schema is additional work for Spark, and it takes some extra timeSchema inference is ...