Hi,I attended Databricks certified associate developer for apache spark 3.0- scala on 09 July 2023(today). At 7.35pm suddenly got a notice stating that due to eye movement away from exam your exam got suspended.I had completed the exam and was at the...
I am applying mosaic's `grid_boundary` method on a spark DataFrame containing a set of `h3_hex_ids`. The geometries returned are not consistent. i.e they could be either `lat, long` or `long, lat`.Here's a sample data```import pyspark.sql.functions a...
Hi All,Yesterday (05.07.2023) databricks experienced a near total outage in the West and North Europe regions. It took the full working day, but they posted an update in the late afternoonstating that a workaround has been put in place and service is...
Dear All, I have resolved the issue by re-installing all libraries and drivers. The reason the drivers were causing me iussues was because my runtime version was updated, and I had no idea. This is probably ehat caused all the issues in the first pla...
Hi, I Have to capture the info when cluster is in hung state for example any cluster is running from last 2 hrs. But it doesn't show any progress. Where can I get hung details.
You can look at the sparkUI of the cluster to get what is happening on the cluster.Debugging with the Apache Spark UI - Azure Databricks | Microsoft Learn
Hi,where do I find the resource mentioned in the latest talks of the Data+AI Summit 2023? In the videos the speaker mention, that there are resources avaible for example in the talk "Best Practices for Setting Up Databricks SQL at Enterprise Scale" (...
Hello, Do some other users have some issues with Databricks today (Clusters, connection, server)?I'm experiencing slowness and failure when running notebooks. My Databricks is on Azure West Europe in Premium.
Hi team,Users may experience slow rendering of notebooks, clusters, and other Databricks user interfaces. Our engineering team is engaged and our status page is updated.https://status.azuredatabricks.net/Some preliminary explanation on Azure website ...
Hi, I use Terraform databricks_job to submit a job with many tasks. IFAIK, each task is a separated Spark Application with a specific applicationId (https://spark.apache.org/docs/3.1.3/api/python/reference/api/pyspark.SparkContext.applicationId.html)...
Hi,I have created my account using my email address on https://community.databricks.com, and my credentials work on https://community.databricks.com.But, when I try login using the same credentials on partner-academy.databricks.com, my login fails. ...
Hi @nikhiljatiwal,Thank you for reaching out!Please submit a ticket to our Training Team here: https://help.databricks.com/s/contact-us?ReqType=training and our team will get back to you shortly.
It’s a wonderful summit! I met so many women in tech during this event, it’s a very good opportunity to exchange thought and ideas with such a divers ppl to modernize Data & AI! I actually join here from Japan, and unfortunately not so many women in ...
Thanks Sakura for dropping by at the Community cove and asking these questions. Together folks like yourself can move the needle and bring in more women to Data and AI space by being inspirational. I am sure you will attract many of us to be in this ...
Hello, Community members! I hope this message finds you well. I wanted to reach out and inquire about your experience with DAIS and how you are absorbing all the valuable information it provides. Additionally, I would love to hear what aspects of Gen...
Dear DBricks Team, Could you kindly look at the Back End Service issues? Thanks a Million.
Hello, My requirement is to created a materialized view (Live table) from a downstream table. The down stream table gets refreshed frequently and hence the materialized view needs to be recalculated. I don't want the pipe line to be in continuous mod...
Hi @Chengcheng I am fine with manual refresh also and have tried "Refresh Table <TableName>" command but it doesn't refreshes the table with new data.The Full refresh command "REFRESH TABLE <tableName> FULL; is giving below error inside notebook.REFR...
I thought this would be a good place to share your favorite talks from the Databricks Data+AI Summit 2023 Heck feel free to share your favorite talk from a different conference or previous summit Today's talk 'Delta-rs, Apache Arrow, Polars, WASM: Is...
I am having an issue where my permissions aren't being passed to workflows. I can use the same cluster to access files in an S3 repo but if I try to do spark-submit or a jar task on workflows I am unable to reach my buckets. I am trying to use Unity ...
Hi Databricks community, I have a blob storage folder that will receive file drops, with 3 files in each distinct drop: ex. Files A1,B1,C1 is one drop, A2,B2,C2 is the next drop. The DLT pipeline I've got setup has a lot of joins and aggregations, cu...
