- 1028 Views
- 1 replies
- 1 kudos
Efficiently Delete/Update/Insert Large Datasets of Records in PostgreSQL from Spark DataFrame
So I am migrating my ETL Process from Pentaho to Databricks I am using pyspark.I have posted all the details here Staging Ground: How to Insert,Update,Delete data using databricks for large records in PostgreSQL from Spark DataFrame - Stack OverflowC...
- 1028 Views
- 1 replies
- 1 kudos
- 1 kudos
Hello @skohade1! The link you provided isn't accessible. Could you please share the relevant details directly here? This will help Community members better understand your scenario.
- 1 kudos
- 520 Views
- 1 replies
- 0 kudos
- 520 Views
- 1 replies
- 0 kudos
- 0 kudos
Hello @mahima13! Could you please share a bit more about the issue? Are you seeing an error when trying to join, or are you unable to locate the session link?
- 0 kudos
- 965 Views
- 2 replies
- 0 kudos
Databricks Billing support
Hi everyone,I dont know if this is the right channel to use for this type of query but I couldn't get Databricks billing support from anywhere, which is why I am posting here. I am trying to contact Databricks sales team or billing support but I am u...
- 965 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi @brycejune, I dont have the support plan, I was only exploring managed MLflow provided by Databricks. Help center however did assign me a Databricks account executive. Hopefully, this might help me with billing.
- 0 kudos
- 920 Views
- 1 replies
- 0 kudos
Resolved! Kryterion suspended my certification exam
Hi Team,I have given the Databricks Certified Associate Developer for Apache Spark - Python Exam on18-05-2025. I was continuously in front of the camera and an alert came and the proctor asked me to show the entire room and bed. And I have shown him ...
- 920 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @vamsikrishna880 Please consider removing your webassessor ID from this post. We handle tickets via our ticket system so that individuals do not need to post their personal contact information in this forum.
- 0 kudos
- 6054 Views
- 10 replies
- 2 kudos
Variables in databricks.yml "include:" - Asset Bundles
HI,We've got an app that we deploy to multiple customers workspaces. We're looking to transition to asset bundles. We would like to structure our resources like: -src/ -resources/ |-- customer_1/ |-- job_1 |-- job_2 |-- customer_2/ |-- job_...
- 6054 Views
- 10 replies
- 2 kudos
- 2 kudos
@_escoto You can achieve this by defining the entire job in the "targets" section for your development target. Resources defined in the top-level "resources" block are deployed to all targets. Resources defined both in the top-level "resources" block...
- 2 kudos
- 583 Views
- 1 replies
- 0 kudos
Data engineer Associate voucher
Hi all, I am new to this databricks and want to learn and do certification for my job search. Is there any free vouchers something like that for exams?
- 583 Views
- 1 replies
- 0 kudos
- 0 kudos
Hello @bindu22! Currently, Databricks isn’t offering any Certification discount promotions, but Learning Festival events are held once per quarter — in January, April, July, and October. Participants receive a 50% off voucher. Details for the next Le...
- 0 kudos
- 2674 Views
- 3 replies
- 0 kudos
Databricks Apps is now available in Public Preview
Databricks Apps is a new way to build and deploy internal data and AI applications is now available in Public Preview.Databricks Apps let developers build native apps using frameworks like Dash, Shiny and Streamlit, enabling data applications for non...
- 2674 Views
- 3 replies
- 0 kudos
- 0 kudos
Databricks Apps is Generally Available as of May 13, 2025. Here is the link to the documentation.
- 0 kudos
- 983 Views
- 1 replies
- 0 kudos
Databricks Bundle Deployment Question
Hello, everyone! I’ve been working on deploying Databricks bundles using Terraform, and I’ve encountered an issue. During the deployment, the Terraform state file seems to reference resources tied to another user, which causes permission errors.I’ve ...
- 983 Views
- 1 replies
- 0 kudos
- 0 kudos
Hey @leandro09 ,My opinion this kind of error is often related to Terraform state still tracking a resource that was created or managed by another user, and your current user doesn’t have permissions to update or read it.terraform state listThis will...
- 0 kudos
- 895 Views
- 1 replies
- 0 kudos
Resolved! My Databrick exam got Suspended.
Hello Databrick Support Team, request ID: #00667456I faced issue while taking my Databrick Certified Associate Developer for Apache Spark exam. This is completely unfair that alerts are coming while taking mid of exam due to blinking of eyes and proc...
- 895 Views
- 1 replies
- 0 kudos
- 0 kudos
Hello @Siddartha01! It looks like this post duplicates the one you recently posted. A response has already been provided to the Original thread. I recommend continuing the discussion in that thread to keep the conversation focused and organised.
- 0 kudos
- 2955 Views
- 0 replies
- 0 kudos
Seeking GitBook Alternatives for Documenting Databricks Pipelines & ML Workflows
We’re scaling our use of Databricks notebooks and ML pipelines, and keeping everything properly documented is becoming a challenge. I’ve used GitBook in past projects, but it’s starting to feel a bit clunky, especially when trying to keep markdown do...
- 2955 Views
- 0 replies
- 0 kudos
- 3849 Views
- 1 replies
- 0 kudos
Unable to create a folder inside DBFS on Community Edition
Im using the Community Edition.Trying to create a storage folder inside DBFS -> Filestore for my datasets. I click on Create, give a folder name, and poof. Nothing. No new folder.Tried refreshing, logging out and logging in. Tried to create folder mu...
- 3849 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi Sachin, For DBFS V1 used in Community Edition, there may be limitations regarding the creation of folders or changes to file structures, particularly under specific paths such as /FileStore, which is commonly used to store datasets, plots, and oth...
- 0 kudos
- 1316 Views
- 3 replies
- 0 kudos
Resolved! Does Unity Catalog support Iceberg?
Hi, I came across this post on querying an AWS Glue table in S3 from Databricks using Unity Catalog: https://medium.com/@sidnahar13/aws-glue-meets-unity-catalog-effortless-data-governance-querying-in-databricks-5138f1e574a4. The post says support for...
- 1316 Views
- 3 replies
- 0 kudos
- 0 kudos
As of now, Databricks does not support non-Spark methods (such as direct REST API calls, JDBC/ODBC connections outside Spark, or other query engines within Databricks) to query S3 Tables directly. Unity Catalog and Databricks’ native cataloging foc...
- 0 kudos
- 9787 Views
- 12 replies
- 13 kudos
Resolved! Uploading local file
 Since, last two day i getting an error called "ERROR OCCURRED WHEN PROCESSING FILE:[OBJECT OBJECT]" While uploading any "csv" or "json" file from my local system but it shows or running my previous file but give error after uploading a new file
- 9787 Views
- 12 replies
- 13 kudos
- 13 kudos
If you are using databricks community edition, the error you are facing is because the file you are trying to upload contains PII or SPII data ( Personally Identifiable Information OR Sensitive Personally Identifiable Information) words like dob, To...
- 13 kudos
- 4060 Views
- 3 replies
- 5 kudos
Getting java.util.concurrent.TimeoutException: Timed out after 15 seconds on community edition
Im using databricks communtiy edition for learning purpose and im whenever im running notebook, im getting:Exception when creating execution context: java.util.concurrent.TimeoutException: Timed out after 15 seconds databricks.I have deleted cluster ...
- 4060 Views
- 3 replies
- 5 kudos
- 5 kudos
@stevieg95 The issue is that, you've run the notebook with old connector connected to your old deleted cluster with same names, when you ran a terminated cluster, you see the error. First delete existing cluster and logout and detach old cluster as b...
- 5 kudos
- 1818 Views
- 6 replies
- 0 kudos
Issue with Multiple Stateful Operations in Databricks Structured Streaming
Hi everyone,I'm working with Databricks structured streaming and have encountered an issue with stateful operations. Below is my pseudo-code: df = df.withWatermark("timestamp", "1 second") df_header = df.withColumn("message_id", F.col("payload.id"))...
- 1818 Views
- 6 replies
- 0 kudos
- 0 kudos
This should according to this blog post basically work, right? However, I'm getting the same errorMultiple Stateful Streaming Operators | Databricks BlogOr am I missing something? rate_df = spark.readStream.format("rate").option("rowsPerSecond", "1")...
- 0 kudos
