Databricks Community

Hi - what error are you getting ? Do you have sufficient permission to drop / delete a table ? 

Hi,This is not an issue, pls understand order of execution of SQL queries. "Order by" clause will always refer to columns selected / displayed (as you are referring as id everywhere I guess there is a confusion).Ambiguous column name exception occurs...

Hi - pls see belowI hope it helps !!

Hi,After trying a lot I could able to see some success , see if this is what you all are looking for :notebook_test.py   (this is python code file)from pyspark.sql import functions as Fdef sum_values(df    return df.agg(F.sum("value")).first()[0]def ...

@nengen  Try using EXPLAIN EXTENDED: This provides a detailed breakdown of the logical and physical plan of a query in Spark SQL.Based on the EXPLAIN EXTENDED output, here are a few things to consider:Broadcast Exchange: If the join causes data skew,...

Hi @filipniziol Thanks for your reply My issue is resolved as my fellow developer ran same commands with different name after sometime it worked successfully.FYI - I was running query in same notebook just in different cells also I was running cells ...

Thanks. We use a job to load data from Kinesis to delta table. I added the spark.databricks.delta.checkpoint.writeFormat parquet spark.databricks.delta.checkpoint.writeStatsAsStruct truein job cluster, but the checkpoints still show different formats...

IMO there is no single best scenario.It depends on the case I would say.  Both have pros and cons.If the difference between teams is really small, views could be a solution.But on the other hand, if you work on massive data, the views first have to b...