Databricks Office HoursRegister for Office Hours to participate in a LIVE Q&A session and receive technical support directly from Databricks experts! Our next event is scheduled for July 27th from 8:00am - 9:00am PT | 3:00pm - 4:00pm GMT.Whether you ...
Hi Guys,Quick thing can we do MergeSchema on update mode in streaming or it is overwrite schema while using update mode .
Hey @Shikher Singh​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Tha...
What are the reasons behind Databricks going for their own driver? What differences are made when switching between the previous Spark driver and the new Databricks driver?Is there any specific document I can look at or just the release notes?Also, w...
Hey @Sriramkumar Thamizharasan​ Hope all is well! Just wanted to check in if you were able to resolve your issue would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from...
Hi have the following case. I want to perform upcert operation. But along with upcert, I want to delete the records which are missing in source table, but present in the target table. You can think it as a master data update.Source table contains a f...
Hey @Krishna Puthran​ Hope you are well. Just wanted to see if you were able to find an answer to your question and would you like to mark an answer as best? It would be really helpful for the other members too.Cheers!
When running Docker for a long time, there are a lot of images in the system. How can I remove all unused Docker images at once safely to free up the storage?In addition, I also want to remove images pulled months ago, So, I'm not asking for removing...
Hey there @william smith​ Hope everything is going great!Does @Prabakar Ammeappin​'s response answer your question? If yes, would you be happy to mark it as best so that other members can find the solution more quickly? Else please let us know if you...
I've created a new job with the new UI / feature enabled. I managed to create one task with a new job cluster successfully but when adding a second task with a new job cluster and trying to save it I received the following error: Error updating [Job...
Hi, I have a similar issue.I'm using the 14 day free trial, setting up the default basic-starter cluster.Then I just followed the basic introduction of dbx with python. https://docs.databricks.com/dev-tools/dbx.htmlCommand Execute is working, but whe...
Hi there,I'm taking a class in Databricks academy on DataBricks SQL and my company dashboard and community dashboard both are missing the SQL dropdown option. How can those taking classes in the academy actually use the tools we are learning about if...
https://community.databricks.com/s/question/0D58Y00008wA5klSAC/flatten-a-complex-json-file-and-load-into-a-delta-tableMyAARPMedicare.com
I want to update one widget based on another widget. It gets updated but the dropdown shows the last selected in the dashboard view, but if I go to the notebook view from the dashboard view it updates. Any help? is it a bug?
It will be attached to the same cluster, and both will see live the same situation (so it will be, in fact, not run two times but one shared)You can use Repos and branches so everyone will get their branch and own notebook.
Hi,AllWhen I check cloud resources, the VPC status is BROKEN.However, the cluster is runnning without any problems.What is the BROKEN state?And how can I get it healthy?Regards.
Hey all, We're trying to analyze the data in a 23 GB JSON file. We're using the basic starter cluster - one node, 2 cpu x 8GB.We can read the JSON file into a spark dataframe and print out the schema but if we try and do any operations that won't c...
Hi there.I can get Databricks cost(dbus) from usage_log. But, how do I get AWS cost information?I want to show Databricks and AWS cost in my Databricks SQL Dashborad.
Hi @Kaniz Fatma​ @Prabakar Ammeappin​ Sorry for the late reply.Every answers was helpful for me! My problem has been solved. Thanks!
How do I configure plot options through the `display` function as code (not through the interactive UI)? Specifically asking since when a notebook is scheduled to run in a databricks job there is no way of configuring the plot type.
I want to create a personal access token for a service principal so that I can use that service principal personal access token in the databricks-connect configure command in an automated build. I followed the instructions from here.https://docs.data...
@Vikas B​ https://docs.databricks.com/dev-tools/api/latest/scim/scim-sp.html#scim-api-20-serviceprincipals let me know if this helps.
I am facing issue in while accessing python data frame in Scala shell and vice versa. I am getting error variable not defined.
The context is not shared between Scala and Python so you won't be able to access the same variables directly. However you can use createOrReplaceTempView to create a temporary view of your dataframe and read it in the other language with read_df = s...
| User | Count |
|---|---|
| 1644 | |
| 793 | |
| 553 | |
| 349 | |
| 287 |
