cancel
Showing results for 
Search instead for 
Did you mean: 
Community Platform Discussions
Connect with fellow community members to discuss general topics related to the Databricks platform, industry trends, and best practices. Share experiences, ask questions, and foster collaboration within the community.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Obulreddy
by New Contributor
  • 3063 Views
  • 3 replies
  • 1 kudos

Unable to access S3 objects from Databricks using IAM access keys in both AWS and Azure Databricks

Hi Team,We are trying to connect to Amazon S3 bucket from both Databricks running on AWS and Azure using IAM access keys directly through Scala code in Notebook and we are facing com.amazonaws.services.s3.model.AmazonS3Exception: Forbidden; with stat...

  • 3063 Views
  • 3 replies
  • 1 kudos
Latest Reply
Anonymous
Not applicable
  • 1 kudos

Hi @Obulreddy  We haven't heard from you since the last response from @KaKa â€‹, and I was checking back to see if her suggestions helped you. Or else, If you have any solution, please share it with the community, as it can be helpful to others.  Also,...

  • 1 kudos
2 More Replies
Sparktaculer
by New Contributor II
  • 2667 Views
  • 4 replies
  • 3 kudos

How to read gcs paths with square barkets?

Hi! I'm trying to read a file using Scala from gcs that has square brackets in the file path.I keep getting the following error:URISyntaxException: Illegal character in path at index 209I tried putting an extra front slash in front of them but it sti...

  • 2667 Views
  • 4 replies
  • 3 kudos
Latest Reply
Sparktaculer
New Contributor II
  • 3 kudos

Hi @Kaniz ! Thank you for your help.However, when I try using you're code I still get an error: "URISyntaxException: Illegal character in path at index "I'm trying to read a txt file. This is the file path: "gs://my-bucket/my Data/sparkTests/GM-1220,...

  • 3 kudos
3 More Replies
Volkan_Gumuskay
by New Contributor III
  • 3211 Views
  • 0 replies
  • 0 kudos

List all delta tables in a database with total size, last snapshot size and user using python/sql

I am trying to list all delta tables in a database and retrieve the following columns: `totalsizeinbyte`, `sizeinbyte` (i.e. the size of last snap shot size) and `created_by` (`lastmodified_by` could also work). Checking online I came across the foll...

  • 3211 Views
  • 0 replies
  • 0 kudos
Avinash_Bevara
by New Contributor III
  • 3212 Views
  • 6 replies
  • 0 kudos

Resolved! Databricks Certified Data Engineer Associate (Version 3) Exam suspended

  Hi Team, My data bricks exam got suspended on 19th July 2023 and it is still in the suspended state. I have raised a support request using the below link https://help.databricks.com/s/contact-us?ReqType=training and my request Id is #00347940  I wo...

  • 3212 Views
  • 6 replies
  • 0 kudos
Latest Reply
APadmanabhan
Databricks Employee
  • 0 kudos

Hi again, I have notified the team that handles these requests, and they will get back to you.I appreciate your patience.

  • 0 kudos
5 More Replies
ehpogue
by New Contributor III
  • 1818 Views
  • 0 replies
  • 0 kudos

Group by using a filter/parameter in SQL dashboard?

Hey all,I'm trying to build a SQL dashboard where the consumer can select a date range, but then can also select whether they want to see the results grouped by day/week/month over that timeframe. I think this is possible for a notebook dashboard by ...

Community Platform Discussions
dashboard
filters
parameters
sql
widgets
  • 1818 Views
  • 0 replies
  • 0 kudos
deficiant_codge
by Contributor II
  • 14918 Views
  • 5 replies
  • 8 kudos

How to run a different python version then the default one on databricks cluster

Hi All, I recently have a use case where i need to run something on the databricks cluster and the bare requirement is that python version shoulde be 3.8 and DBR version should be 11+. I am confused on how to handle this. Is there anyway in which we ...

  • 14918 Views
  • 5 replies
  • 8 kudos
Latest Reply
Anonymous
Not applicable
  • 8 kudos

Hi @deficiant_codge  Hope everything is going great. Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us...

  • 8 kudos
4 More Replies
Benedetta
by New Contributor III
  • 2927 Views
  • 0 replies
  • 0 kudos

SQL Merge not adding new columns to delta table

SQL Merge not adding new columns to delta table error is: cannot resolve Target AuditUpdateDate in UPDATE clause given columns in Sourcethis config is set: spark.conf.set("spark.databricks.delta.schema.autoMerge.enabled",True)We're using DBR 12.2 LTS...

Benedetta_0-1689344912359.png
  • 2927 Views
  • 0 replies
  • 0 kudos
kll
by New Contributor III
  • 2330 Views
  • 4 replies
  • 1 kudos

SparkException: Job aborted due to stage failure when attempting to run grid_pointascellid

I am attempting to apply Mosaic's `grid_pointascellid` method on a spark dataframe with `lat`, `lon` columns.```import pyspark.sql.functions as F# Create a Spark DataFrame with a lat and lon columndf = spark.createDataFrame([("point1", 10.0, 20.0),("...

  • 2330 Views
  • 4 replies
  • 1 kudos
Latest Reply
Tharun-Kumar
Databricks Employee
  • 1 kudos

@kll This error appears because the function grid_pointascellid expects a Double type column and a Decimal column type was provided as input.To overcome this, before you apply the grid_pointascellid, I would recommend casting the columns lat and lon....

  • 1 kudos
3 More Replies
gilo12
by New Contributor III
  • 2012 Views
  • 2 replies
  • 2 kudos

Bug - Cannot create table The associated location is not empty and also not a Delta table.

I am getting the error:Cannot create table ('`hive_metastore`.`MY_SCHEMA`.`MY_TABLE`'). The associated location ('dbfs:/user/hive/warehouse/my_schema.db/my_table') is not empty and also not a Delta table.When running drop table 'hive_metastore`.`MY_S...

  • 2012 Views
  • 2 replies
  • 2 kudos
Latest Reply
Anonymous
Not applicable
  • 2 kudos

Hi @gilo12  Thank you for posting your question in our community! We are happy to assist you. To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers your ...

  • 2 kudos
1 More Replies
joao_vnb
by New Contributor III
  • 8065 Views
  • 3 replies
  • 0 kudos

Give permissions to add comments on data explorer

Hi everyone!I'm facing this issue on databricks workspace.I do have permissions to add comments on data explorer, but some members of my team don't have those permissions.Where can I configure this and give them permission? 

joao_vnb_1-1689087557966.png
  • 8065 Views
  • 3 replies
  • 0 kudos
Latest Reply
Prabakar
Databricks Employee
  • 0 kudos

To grant the permissions on the same page that you have shared, click the permissions tab, and select grant.then from the dropdown list select the user or group and give the respective permissions. 

  • 0 kudos
2 More Replies
databricks2k6
by New Contributor II
  • 1575 Views
  • 2 replies
  • 0 kudos

community edition

community edition is available?I tried to login but it is redirecting me differnt pages.

  • 1575 Views
  • 2 replies
  • 0 kudos
Latest Reply
databricks2k6
New Contributor II
  • 0 kudos

Hi ,Please note that community and  is completely fiffernt objectives but you are refering community for disscussion foroum.Unlike the Databricks Free Trial, Community Edition doesn’t require that you have your own cloud account or supply cloud compu...

  • 0 kudos
1 More Replies
ManishaSethi
by New Contributor
  • 1435 Views
  • 2 replies
  • 0 kudos

Generative AI Badge not received ID: E-1K6NE1

Passed the test with 90%. , nothing gets reflected under badge section at databricks portal or credentials.databricks.com.Request support in getting the badge.

  • 1435 Views
  • 2 replies
  • 0 kudos
Latest Reply
Anonymous
Not applicable
  • 0 kudos

Hi @ManishaSethi , Thank you for posting your question in our community! We are happy to assist you. To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answer...

  • 0 kudos
1 More Replies
Paolo
by New Contributor
  • 970 Views
  • 0 replies
  • 0 kudos

Bug - task retry fails to load cluster dependencies

Hi,I'm currently facing an issue with task retries on a structured streaming set to have unlimited retries. Our job crashes frequently due to out of memory problems, so we set the task retry limit to -1  (unlimited) as a workaround. This is also the ...

Community Platform Discussions
Bug
Cluster initialization
Dependency
structured streaming
  • 970 Views
  • 0 replies
  • 0 kudos

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group
Top Kudoed Authors