- 571 Views
- 0 replies
- 0 kudos
Hi all
great to be here
- 571 Views
- 0 replies
- 0 kudos
Forum Posts
- 15759 Views
- 5 replies
- 8 kudos
How to run a different python version then the default one on databricks cluster
Hi All, I recently have a use case where i need to run something on the databricks cluster and the bare requirement is that python version shoulde be 3.8 and DBR version should be 11+. I am confused on how to handle this. Is there anyway in which we ...
- 15759 Views
- 5 replies
- 8 kudos
Latest Reply
- 8 kudos
Hi @deficiant_codge Hope everything is going great. Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us...
- 8 kudos
- 2996 Views
- 0 replies
- 0 kudos
SQL Merge not adding new columns to delta table
SQL Merge not adding new columns to delta table error is: cannot resolve Target AuditUpdateDate in UPDATE clause given columns in Sourcethis config is set: spark.conf.set("spark.databricks.delta.schema.autoMerge.enabled",True)We're using DBR 12.2 LTS...
- 2996 Views
- 0 replies
- 0 kudos
- 2388 Views
- 4 replies
- 1 kudos
SparkException: Job aborted due to stage failure when attempting to run grid_pointascellid
I am attempting to apply Mosaic's `grid_pointascellid` method on a spark dataframe with `lat`, `lon` columns.```import pyspark.sql.functions as F# Create a Spark DataFrame with a lat and lon columndf = spark.createDataFrame([("point1", 10.0, 20.0),("...
- 2388 Views
- 4 replies
- 1 kudos
Latest Reply
- 1 kudos
@kll This error appears because the function grid_pointascellid expects a Double type column and a Decimal column type was provided as input.To overcome this, before you apply the grid_pointascellid, I would recommend casting the columns lat and lon....
- 1 kudos
- 2069 Views
- 2 replies
- 2 kudos
Bug - Cannot create table The associated location is not empty and also not a Delta table.
I am getting the error:Cannot create table ('`hive_metastore`.`MY_SCHEMA`.`MY_TABLE`'). The associated location ('dbfs:/user/hive/warehouse/my_schema.db/my_table') is not empty and also not a Delta table.When running drop table 'hive_metastore`.`MY_S...
- 2069 Views
- 2 replies
- 2 kudos
Latest Reply
- 2 kudos
Hi @gilo12 Thank you for posting your question in our community! We are happy to assist you. To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers your ...
- 2 kudos
- 8311 Views
- 3 replies
- 0 kudos
Give permissions to add comments on data explorer
Hi everyone!I'm facing this issue on databricks workspace.I do have permissions to add comments on data explorer, but some members of my team don't have those permissions.Where can I configure this and give them permission?
- 8311 Views
- 3 replies
- 0 kudos
Latest Reply
- 0 kudos
To grant the permissions on the same page that you have shared, click the permissions tab, and select grant.then from the dropdown list select the user or group and give the respective permissions.
- 0 kudos
- 1617 Views
- 2 replies
- 0 kudos
community edition
community edition is available?I tried to login but it is redirecting me differnt pages.
- 1617 Views
- 2 replies
- 0 kudos
Latest Reply
- 0 kudos
Hi ,Please note that community and is completely fiffernt objectives but you are refering community for disscussion foroum.Unlike the Databricks Free Trial, Community Edition doesn’t require that you have your own cloud account or supply cloud compu...
- 0 kudos
- 752 Views
- 0 replies
- 0 kudos
How to access model endpoint management APIs from workflow tasks?
I have created a model serving endpoint in Databricks. I'd like to now update this endpoint programmatically from a workflow task.Following instructions on this page:https://docs.databricks.com/machine-learning/model-serving/create-manage-serving-end...
- 752 Views
- 0 replies
- 0 kudos
- 1488 Views
- 2 replies
- 0 kudos
Generative AI Badge not received ID: E-1K6NE1
Passed the test with 90%. , nothing gets reflected under badge section at databricks portal or credentials.databricks.com.Request support in getting the badge.
- 1488 Views
- 2 replies
- 0 kudos
Latest Reply
- 0 kudos
Hi @ManishaSethi , Thank you for posting your question in our community! We are happy to assist you. To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answer...
- 0 kudos
- 997 Views
- 0 replies
- 0 kudos
Bug - task retry fails to load cluster dependencies
Hi,I'm currently facing an issue with task retries on a structured streaming set to have unlimited retries. Our job crashes frequently due to out of memory problems, so we set the task retry limit to -1 (unlimited) as a workaround. This is also the ...
- 997 Views
- 0 replies
- 0 kudos
- 805 Views
- 0 replies
- 1 kudos
Databricks Exam got suspended
Hi,I attended Databricks certified associate developer for apache spark 3.0- scala on 09 July 2023(today). At 7.35pm suddenly got a notice stating that due to eye movement away from exam your exam got suspended.I had completed the exam and was at the...
- 805 Views
- 0 replies
- 1 kudos
- 705 Views
- 0 replies
- 0 kudos
Mosaic's grid_boundary method returns inconsistent geometries
I am applying mosaic's `grid_boundary` method on a spark DataFrame containing a set of `h3_hex_ids`. The geometries returned are not consistent. i.e they could be either `lat, long` or `long, lat`.Here's a sample data```import pyspark.sql.functions a...
- 705 Views
- 0 replies
- 0 kudos
- 2569 Views
- 2 replies
- 0 kudos
Resolved! Library and Driver issues following Databricks outage on 05.07.2023 (Azure)
Hi All,Yesterday (05.07.2023) databricks experienced a near total outage in the West and North Europe regions. It took the full working day, but they posted an update in the late afternoonstating that a workaround has been put in place and service is...
- 2569 Views
- 2 replies
- 0 kudos
Latest Reply
- 0 kudos
Dear All, I have resolved the issue by re-installing all libraries and drivers. The reason the drivers were causing me iussues was because my runtime version was updated, and I had no idea. This is probably ehat caused all the issues in the first pla...
- 0 kudos
- 1270 Views
- 1 replies
- 1 kudos
Resolved! Find out the cluster hung state in azure databricks
Hi, I Have to capture the info when cluster is in hung state for example any cluster is running from last 2 hrs. But it doesn't show any progress. Where can I get hung details.
- 1270 Views
- 1 replies
- 1 kudos
Latest Reply
- 1 kudos
You can look at the sparkUI of the cluster to get what is happening on the cluster.Debugging with the Apache Spark UI - Azure Databricks | Microsoft Learn
- 1 kudos
- 667 Views
- 0 replies
- 0 kudos
Resources for Data+AI Summit Talks
Hi,where do I find the resource mentioned in the latest talks of the Data+AI Summit 2023? In the videos the speaker mention, that there are resources avaible for example in the talk "Best Practices for Setting Up Databricks SQL at Enterprise Scale" (...
- 667 Views
- 0 replies
- 0 kudos
