- 2166 Views
- 1 replies
- 1 kudos
Databricks notebook exceptions into Service Now
Hello Community members,I am looking for options for redirecting the Databricks notebook raised except within exception block to be redirected to ServiceNowIs there a way the connection can be made directly from the notebook?Looking for suggestions. ...
- 2166 Views
- 1 replies
- 1 kudos
- 1 kudos
Thank you for the solution, I will definitely try this and share to the community if this works.
- 1 kudos
- 4131 Views
- 3 replies
- 4 kudos
configuration for Job Queueing in Terraform
When defining the databricks_job resource in Terraform , we are trying to enable Job Queueing flag for the job. However, from the Terraform Provider docs, we are not able to find any config related to queuing. Is there a different method to configure...
- 4131 Views
- 3 replies
- 4 kudos
- 4 kudos
I've created a Feature Request for this in Github - https://github.com/databricks/terraform-provider-databricks/issues/2531
- 4 kudos
- 15881 Views
- 3 replies
- 2 kudos
Resolved! AnalysisException: [RequestId=... ErrorClass=INVALID_PARAMETER_VALUE] Missing cloud file system scheme
Hi community,i get an analysis exception when executing following code in a notebook using a personal compute cluster. Seems to be an issue with permission but I am logged in with my admin account. Any help would be appreciated. USE CATALOG catalog; ...
- 15881 Views
- 3 replies
- 2 kudos
- 2 kudos
I was having the same issue because I was trying to set the location with the absolute path, just like you did.I solved it by creating an external location, then copying the URL and putting it into the location of the path options.
- 2 kudos
- 19767 Views
- 6 replies
- 6 kudos
In what circumstances are both UAT/DEV and PROD environments actually necessary?
I wanted to ask this Q yesterday in the Q&A session with Mohan Mathews, but didn't get around to it (@Kaniz Fatma​ do you know his handle here so I can tag him?)We (and most development teams) have two environments: UAT/DEV and PROD. For those that d...
- 19767 Views
- 6 replies
- 6 kudos
- 6 kudos
Hi @Oliver Angelil​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Tha...
- 6 kudos
- 6954 Views
- 3 replies
- 3 kudos
Resolved! Order of delta table after read not as expected
Dear Databricks Community,I am performing three consecutive 'append' writes to a delta table, whereas the first append creates the table. Each append consists of two rows, which are ordered by column 'id' (see example in the attached screenshot). Whe...
- 6954 Views
- 3 replies
- 3 kudos
- 3 kudos
Thanks a lot @Lakshay and @Tharun-Kumar for your valued contributions!
- 3 kudos
- 2924 Views
- 2 replies
- 3 kudos
Job aborted due to stage failure. Relative path in absolute URI
Hello Team we have frequently data bricks job failure with following message , any help would be appreciated Job aborted due to stage failure. Relative path in absolute URI
- 2924 Views
- 2 replies
- 3 kudos
- 3 kudos
@DB_PROD_Molina One of the reasons this error shows up is due to file path/name containing special characters in it. If that is the case, could you rename your file to have the special characters removed.
- 3 kudos
- 6351 Views
- 3 replies
- 3 kudos
Resolved! PARSE_SYNTAX_ERROR creating view from CSV
Hello, if i run this code: %sqlCREATE OR REPLACE VIEW esprosilver.xxx.encuestas_talleresASSELECT * FROM CSV.`abfss://landing@esproanalyticscenterdl.dfs.core.windows.net/oracle-dwh/encuestas_talleres/encuestas_talleres.csv` It creates the view in unit...
- 6351 Views
- 3 replies
- 3 kudos
- 3715 Views
- 0 replies
- 0 kudos
Running dbt macro inside Databricks notebook as a part of Workflow
So.. I have Databricks and I'm using dbt-databricks adapter to run dbt. Everything works fine but for some reasons I need to run some stuff inside of notebook. That still worked fine for some time but after upgrading to dbt-databricks 1.5.5 something...
- 3715 Views
- 0 replies
- 0 kudos
- 3190 Views
- 2 replies
- 1 kudos
Resolved! Serverless clusters not starting
Hello, I am trying to launch a serverless data warehouse, it used to work fine before but for some reason it no longer works. I tried creating a brand new serverless cluster and I get the same result. I am the creator of the clusters, and a workspace...
- 3190 Views
- 2 replies
- 1 kudos
- 1 kudos
For anyone interested in the status page link : https://status.azuredatabricks.net/
- 1 kudos
- 12306 Views
- 6 replies
- 3 kudos
Resolved! Enabling Adaptive Query Execution and Cost-Based Optimizer in Structured Streaming foreachBatch
Dear Databricks community,I am using Spark Structured Streaming to move data from silver to gold in an ETL fashion. The source stream is the change data feed of a Delta table in silver. The streaming dataframe is transformed and joined with a couple ...
- 12306 Views
- 6 replies
- 3 kudos
- 3 kudos
It's not recommended to have AQE on a Streaming query for the same reason you shared in the description. It has been documented here
- 3 kudos
- 3525 Views
- 1 replies
- 2 kudos
Is there a way to avoid using EBS drives on workers with local NVMe SSD?
The Databricks on AWS docs claim that 30G + 150G EBS drives are mounter to every node by default. But if I use instance type like r5d.2xlarge, it already has local disk so I want to avoid mounting the 150G EBS drive to it. Is there a way to do it?We ...
- 3525 Views
- 1 replies
- 2 kudos
- 2 kudos
Hey Ivan, did you find a way to do this?
- 2 kudos
- 2665 Views
- 2 replies
- 0 kudos
Work around for The method `pd.groupby.GroupBy.prod()` is not implemented yet.
0I have a database with two columns: name (str) and probability (float).I am running this command:df[['name','probability']].groupby('name').prod()on a Databricks (runtime 7.3) notebook and df is a koalas dataframe.The error I get is:PandasNotImpleme...
- 2665 Views
- 2 replies
- 0 kudos
- 2668 Views
- 0 replies
- 0 kudos
java.time.LocalDate exception when a date column is used with "IN" operator in replace where clause
Hi, we recently made an upgrade to our Databricks warehouse, transitioning from SQL Classic to SQL PRO. However, we encountered the following error message when attempting to execute the "INSERT INTO" table query with a "REPLACE WHERE" predicate that...
- 2668 Views
- 0 replies
- 0 kudos
- 2300 Views
- 1 replies
- 0 kudos
Databricks-connect VSCode debugging pandas_api not working
HiI am using the databricks extension on VSCode and am running against an issue since two days, prior it worked fine. I receive an error when I want to use Pandas-on-Spark during debugging.from databricks.connect import DatabricksSession spark = Data...
- 2300 Views
- 1 replies
- 0 kudos
- 0 kudos
Additional info:It seems that the issue comes from the 1.1.0 version of the databricks extension in VSCode.Downgrading to 1.0.0 solves my issue.
- 0 kudos
- 5142 Views
- 1 replies
- 2 kudos
SQL expr undefined function 'LEN'
Getting this error message on our production cluster when I run a notebook that uses the SQL expr function that call the LEN() funciton example code:df = df.withColumn("POL", expr("CASE WHEN SRC_SYSTEM = 'X' THEN CONCAT('08' , SUBSTRING(POL, 3, LEN(P...
- 5142 Views
- 1 replies
- 2 kudos
- 2 kudos
@PhillT There's no "LEN" function. You should use "LENGTH" instead.https://spark.apache.org/docs/2.3.0/api/sql/index.html#length
- 2 kudos
| User | Count |
|---|---|
| 1644 | |
| 793 | |
| 581 | |
| 349 | |
| 287 |
