Databricks Community is an open-source platform for data enthusiasts and professionals to discuss, share insights, and collaborate on everything related to Databricks. Members can ask questions, share knowledge, and support each other in an environment that ensures respectful interactions.
We are currently using Spark as our SQL engine with Thrift Server but are evaluating Databricks Serverless SQL as a potential alternative. We have a few specific questions: Does Databricks Serverle...
Hi Everyone, I have been trying to use autoloader with foreach so that I could able to use merge into in databricks, but while using I have been getting below error. error-Found error inside ...
I have 2 tasks (T1 &T2) that run in branch B1 of Repo1. I have created a new task (depends on T2 ) which points to a different branch B2 of same Repo1. Is it possible to run them in the s...
Hi everyone! I hope you’re all doing well. I’ve recently completed my Mount point in unity catalog - Databricks Community - 79478 projects in the cloud, but now I’m facing some frustrating issues. Af...
I'm trying to perform a MERGE between two tables (customers and customers_update) using Spark SQL, but I’m encountering an internal error during the planning phase. The error message suggests it migh...
Hello, I am new to learning about Azure Databricks and decided to take the free online training to learn about it. When I try to access the videos, they will not work for me. I keep seeing a m...
Hi @Sujitha , @Rishabh_Tiwari , I wanted to bring to your attention that whenever I’m writing content on Databricks, I often encounter errors due to invalid HTML. Additionally, some terms ...
i have come accross a documentation on asset bundles long back whcih states that when you type databricks bundle init it gives us option to choose a project type. But i see the below error when i d...
...f them works
repartition_df.write.format("delta").mode("overwrite").option("overwriteSchema", "true").saveAsTable(output_table)
chunked_df.unpersist()