Hi,I'm currently facing an issue with task retries on a structured streaming set to have unlimited retries. Our job crashes frequently due to out of memory problems, so we set the task retry limit to -1 (unlimited) as a workaround. This is also the ...
Hi Everyone,I am trying to get spark context by running SparkSession builder command in a .py file and when i try to run it directl it works but if i got to a notebook and in a cell if i use %sh command to run python "myfilename" it gives an error th...
Hi,I attended Databricks certified associate developer for apache spark 3.0- scala on 09 July 2023(today). At 7.35pm suddenly got a notice stating that due to eye movement away from exam your exam got suspended.I had completed the exam and was at the...
We are thrilled to introduce a dedicated "Get Started with Databricks" space on Databricks Community for new users and curious data practitioners not yet on Databricks! In this space, we are bringing together the best of resources to ensure new and c...
Hi..I would like to know if anyone knows how to send personalized emails from databricks or if there is an example that you can replicate it would be very helpful.Greetings to each one.
Is there any way to get the functionality of databricks repos without using a provider? i.e. Databricks would be the provider. I would like to get the benefits of git operations in databricks but without linking my repository to an external source. I...
I'm afraid this isn't a supported feature today. Databricks Repos acts as a conduit or client between the workspace and a configured Git provider. No hosting functionality is available.
I am applying mosaic's `grid_boundary` method on a spark DataFrame containing a set of `h3_hex_ids`. The geometries returned are not consistent. i.e they could be either `lat, long` or `long, lat`.Here's a sample data```import pyspark.sql.functions a...
Hi All,Yesterday (05.07.2023) databricks experienced a near total outage in the West and North Europe regions. It took the full working day, but they posted an update in the late afternoonstating that a workaround has been put in place and service is...
Dear All, I have resolved the issue by re-installing all libraries and drivers. The reason the drivers were causing me iussues was because my runtime version was updated, and I had no idea. This is probably ehat caused all the issues in the first pla...
I see that databricks supports connecting the unity catalog using hive metastore API (https://www.databricks.com/blog/extending-databricks-unity-catalog-open-apache-hive-metastore-api). Where to find hive metastore thrift uri `hive.metastore.uri` to ...
I am asked to rename the Catalog that I created since it is not as per the guidelines laid out lately so wondering how to rename it. Could you please let me know the procedure at the earliest? Regards~Suresh
kqxosoonline là nơi cung cấp những tin tức mới nhất về xổ số từ các nguồn tin chính thống. Trang web cập nhật liên tục những thông tin, kết quả xổ số, dự đoán… chính xác và nhanh chóng. kqxosoonline thành lập từ năm 2018 và nhanh chóng trở thành cái ...
Hi, I Have to capture the info when cluster is in hung state for example any cluster is running from last 2 hrs. But it doesn't show any progress. Where can I get hung details.
You can look at the sparkUI of the cluster to get what is happening on the cluster.Debugging with the Apache Spark UI - Azure Databricks | Microsoft Learn
Hi,where do I find the resource mentioned in the latest talks of the Data+AI Summit 2023? In the videos the speaker mention, that there are resources avaible for example in the talk "Best Practices for Setting Up Databricks SQL at Enterprise Scale" (...
Hello, Do some other users have some issues with Databricks today (Clusters, connection, server)?I'm experiencing slowness and failure when running notebooks. My Databricks is on Azure West Europe in Premium.
Hi team,Users may experience slow rendering of notebooks, clusters, and other Databricks user interfaces. Our engineering team is engaged and our status page is updated.https://status.azuredatabricks.net/Some preliminary explanation on Azure website ...
Hi, I use Terraform databricks_job to submit a job with many tasks. IFAIK, each task is a separated Spark Application with a specific applicationId (https://spark.apache.org/docs/3.1.3/api/python/reference/api/pyspark.SparkContext.applicationId.html)...
Databricks is a unified set of tools for building, deploying, sharing, and maintaining enterprise-grade data solutions at scale. The Azure Databricks Lakehouse Platform integrates with cloud storage and security in your cloud account, and manages and...
