- 3063 Views
- 3 replies
- 1 kudos
Unable to access S3 objects from Databricks using IAM access keys in both AWS and Azure Databricks
Hi Team,We are trying to connect to Amazon S3 bucket from both Databricks running on AWS and Azure using IAM access keys directly through Scala code in Notebook and we are facing com.amazonaws.services.s3.model.AmazonS3Exception: Forbidden; with stat...
- 3063 Views
- 3 replies
- 1 kudos
- 1 kudos
Hi @Obulreddy We haven't heard from you since the last response from @KaKa , and I was checking back to see if her suggestions helped you. Or else, If you have any solution, please share it with the community, as it can be helpful to others. Also,...
- 1 kudos
- 2667 Views
- 4 replies
- 3 kudos
How to read gcs paths with square barkets?
Hi! I'm trying to read a file using Scala from gcs that has square brackets in the file path.I keep getting the following error:URISyntaxException: Illegal character in path at index 209I tried putting an extra front slash in front of them but it sti...
- 2667 Views
- 4 replies
- 3 kudos
- 3 kudos
Hi @Kaniz ! Thank you for your help.However, when I try using you're code I still get an error: "URISyntaxException: Illegal character in path at index "I'm trying to read a txt file. This is the file path: "gs://my-bucket/my Data/sparkTests/GM-1220,...
- 3 kudos
- 3211 Views
- 0 replies
- 0 kudos
List all delta tables in a database with total size, last snapshot size and user using python/sql
I am trying to list all delta tables in a database and retrieve the following columns: `totalsizeinbyte`, `sizeinbyte` (i.e. the size of last snap shot size) and `created_by` (`lastmodified_by` could also work). Checking online I came across the foll...
- 3211 Views
- 0 replies
- 0 kudos
- 3212 Views
- 6 replies
- 0 kudos
Resolved! Databricks Certified Data Engineer Associate (Version 3) Exam suspended
Hi Team, My data bricks exam got suspended on 19th July 2023 and it is still in the suspended state. I have raised a support request using the below link https://help.databricks.com/s/contact-us?ReqType=training and my request Id is #00347940 I wo...
- 3212 Views
- 6 replies
- 0 kudos
- 0 kudos
Hi again, I have notified the team that handles these requests, and they will get back to you.I appreciate your patience.
- 0 kudos
- 1818 Views
- 0 replies
- 0 kudos
Group by using a filter/parameter in SQL dashboard?
Hey all,I'm trying to build a SQL dashboard where the consumer can select a date range, but then can also select whether they want to see the results grouped by day/week/month over that timeframe. I think this is possible for a notebook dashboard by ...
- 1818 Views
- 0 replies
- 0 kudos
- 554 Views
- 0 replies
- 0 kudos
Hi all
great to be here
- 554 Views
- 0 replies
- 0 kudos
- 14918 Views
- 5 replies
- 8 kudos
How to run a different python version then the default one on databricks cluster
Hi All, I recently have a use case where i need to run something on the databricks cluster and the bare requirement is that python version shoulde be 3.8 and DBR version should be 11+. I am confused on how to handle this. Is there anyway in which we ...
- 14918 Views
- 5 replies
- 8 kudos
- 8 kudos
Hi @deficiant_codge Hope everything is going great. Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us...
- 8 kudos
- 2927 Views
- 0 replies
- 0 kudos
SQL Merge not adding new columns to delta table
SQL Merge not adding new columns to delta table error is: cannot resolve Target AuditUpdateDate in UPDATE clause given columns in Sourcethis config is set: spark.conf.set("spark.databricks.delta.schema.autoMerge.enabled",True)We're using DBR 12.2 LTS...
- 2927 Views
- 0 replies
- 0 kudos
- 2330 Views
- 4 replies
- 1 kudos
SparkException: Job aborted due to stage failure when attempting to run grid_pointascellid
I am attempting to apply Mosaic's `grid_pointascellid` method on a spark dataframe with `lat`, `lon` columns.```import pyspark.sql.functions as F# Create a Spark DataFrame with a lat and lon columndf = spark.createDataFrame([("point1", 10.0, 20.0),("...
- 2330 Views
- 4 replies
- 1 kudos
- 1 kudos
@kll This error appears because the function grid_pointascellid expects a Double type column and a Decimal column type was provided as input.To overcome this, before you apply the grid_pointascellid, I would recommend casting the columns lat and lon....
- 1 kudos
- 2012 Views
- 2 replies
- 2 kudos
Bug - Cannot create table The associated location is not empty and also not a Delta table.
I am getting the error:Cannot create table ('`hive_metastore`.`MY_SCHEMA`.`MY_TABLE`'). The associated location ('dbfs:/user/hive/warehouse/my_schema.db/my_table') is not empty and also not a Delta table.When running drop table 'hive_metastore`.`MY_S...
- 2012 Views
- 2 replies
- 2 kudos
- 2 kudos
Hi @gilo12 Thank you for posting your question in our community! We are happy to assist you. To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers your ...
- 2 kudos
- 8065 Views
- 3 replies
- 0 kudos
Give permissions to add comments on data explorer
Hi everyone!I'm facing this issue on databricks workspace.I do have permissions to add comments on data explorer, but some members of my team don't have those permissions.Where can I configure this and give them permission?
- 8065 Views
- 3 replies
- 0 kudos
- 0 kudos
To grant the permissions on the same page that you have shared, click the permissions tab, and select grant.then from the dropdown list select the user or group and give the respective permissions.
- 0 kudos
- 1575 Views
- 2 replies
- 0 kudos
community edition
community edition is available?I tried to login but it is redirecting me differnt pages.
- 1575 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi ,Please note that community and is completely fiffernt objectives but you are refering community for disscussion foroum.Unlike the Databricks Free Trial, Community Edition doesn’t require that you have your own cloud account or supply cloud compu...
- 0 kudos
- 738 Views
- 0 replies
- 0 kudos
How to access model endpoint management APIs from workflow tasks?
I have created a model serving endpoint in Databricks. I'd like to now update this endpoint programmatically from a workflow task.Following instructions on this page:https://docs.databricks.com/machine-learning/model-serving/create-manage-serving-end...
- 738 Views
- 0 replies
- 0 kudos
- 1435 Views
- 2 replies
- 0 kudos
Generative AI Badge not received ID: E-1K6NE1
Passed the test with 90%. , nothing gets reflected under badge section at databricks portal or credentials.databricks.com.Request support in getting the badge.
- 1435 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi @ManishaSethi , Thank you for posting your question in our community! We are happy to assist you. To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answer...
- 0 kudos
- 970 Views
- 0 replies
- 0 kudos
Bug - task retry fails to load cluster dependencies
Hi,I'm currently facing an issue with task retries on a structured streaming set to have unlimited retries. Our job crashes frequently due to out of memory problems, so we set the task retry limit to -1 (unlimited) as a workaround. This is also the ...
- 970 Views
- 0 replies
- 0 kudos
