Hello,I've got a legacy workspace (not E2) and I am trying to move my cluster scoped init script to the workspace area (from DBFS). It doesn't seem to be possible to store a shell script in the workspace area (Accepted formats: .dbc, .scala, .py, .sq...
I want to host databricks on AWS. I want to know if we create databricks on top of AWS, will it be created in same account's VPC or will it be created out of my AWS account?If it is going to be created in my account, will it create a new VPC for me?T...
Hi, If you want to know more about the how to properly setup the databricks on top of AWS. I would really recommend to do the AWS platform administrator course of Databricks. In this everything is explained what you need to know. Hopes this helps.Kin...
Hello,I have been connecting to Databricks via Alteryx. It works fine when our cluster is active, but returns a 503 Service Unavailable error if the Cluster is inactive/starting up. I have previously reached out to Alteryx, but they have told me this...
I should have mentioned in the original post, we are using Microsoft Azure and a Simba Spark ODBC Driver.
When executing through a Databricks cluster with credential passthrough enabled, I wish to obtain supplementary file attributes in ADLS, such as the file's last modified time, which are currently unavailable in the databricks dbutils.fs.ls function.W...
There are no points displayed in Databricks new community page. Is it the same for all or only for me if I have done something wrong.
same concerns with you. on my account also didnot find where place display how many point in my account
Hi Team,We are trying to connect to Amazon S3 bucket from both Databricks running on AWS and Azure using IAM access keys directly through Scala code in Notebook and we are facing com.amazonaws.services.s3.model.AmazonS3Exception: Forbidden; with stat...
Hi @Obulreddy We haven't heard from you since the last response from @KaKa , and I was checking back to see if her suggestions helped you. Or else, If you have any solution, please share it with the community, as it can be helpful to others. Also,...
Hi! I'm trying to read a file using Scala from gcs that has square brackets in the file path.I keep getting the following error:URISyntaxException: Illegal character in path at index 209I tried putting an extra front slash in front of them but it sti...
Hi @Kaniz ! Thank you for your help.However, when I try using you're code I still get an error: "URISyntaxException: Illegal character in path at index "I'm trying to read a txt file. This is the file path: "gs://my-bucket/my Data/sparkTests/GM-1220,...
I am trying to list all delta tables in a database and retrieve the following columns: `totalsizeinbyte`, `sizeinbyte` (i.e. the size of last snap shot size) and `created_by` (`lastmodified_by` could also work). Checking online I came across the foll...
Hi Team, My data bricks exam got suspended on 19th July 2023 and it is still in the suspended state. I have raised a support request using the below link https://help.databricks.com/s/contact-us?ReqType=training and my request Id is #00347940 I wo...
Hi again, I have notified the team that handles these requests, and they will get back to you.I appreciate your patience.
Hey all,I'm trying to build a SQL dashboard where the consumer can select a date range, but then can also select whether they want to see the results grouped by day/week/month over that timeframe. I think this is possible for a notebook dashboard by ...
great to be here
Hi All, I recently have a use case where i need to run something on the databricks cluster and the bare requirement is that python version shoulde be 3.8 and DBR version should be 11+. I am confused on how to handle this. Is there anyway in which we ...
Hi @deficiant_codge Hope everything is going great. Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us...
SQL Merge not adding new columns to delta table error is: cannot resolve Target AuditUpdateDate in UPDATE clause given columns in Sourcethis config is set: spark.conf.set("spark.databricks.delta.schema.autoMerge.enabled",True)We're using DBR 12.2 LTS...
I am attempting to apply Mosaic's `grid_pointascellid` method on a spark dataframe with `lat`, `lon` columns.```import pyspark.sql.functions as F# Create a Spark DataFrame with a lat and lon columndf = spark.createDataFrame([("point1", 10.0, 20.0),("...
@kll This error appears because the function grid_pointascellid expects a Double type column and a Decimal column type was provided as input.To overcome this, before you apply the grid_pointascellid, I would recommend casting the columns lat and lon....
I am getting the error:Cannot create table ('`hive_metastore`.`MY_SCHEMA`.`MY_TABLE`'). The associated location ('dbfs:/user/hive/warehouse/my_schema.db/my_table') is not empty and also not a Delta table.When running drop table 'hive_metastore`.`MY_S...
Hi @gilo12 Thank you for posting your question in our community! We are happy to assist you. To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers your ...
