Databricks

Raymond_Garcia · 11-14-2022

Hello, I have an autoloader code and it is pretty standard, we have this variable file path that points to an S3 bucket. example #2 executed successfully and example 1 throws an exception.it seems like source 1 always throws an exception whereas sour...

Raymond_Garcia · 10-17-2022

I have like 5 maven libraries, and with all of them, I have the same issue with Job or Notebooks. How much do I have to wait? is there another solution? Thank you very much!

Raymond_Garcia · 07-05-2022

Hello I have a databricks question I was not able to answer myselfI have this queryselect count(*) from tablewhere object[0].value is not null and object[0].value.value1 = "s"and created_year = 2022 and created_month = 7 and created_day = 4you can se...

Raymond_Garcia · 06-24-2022

Hello, I have a data bricks question. A Dataframe job that writes in an s3 bucket usually takes 8 minutes to finish, but now it takes from 8 to 9 hours to complete. Does anybody have some clues about this behavior?the data frame size is about 300 or ...

Raymond_Garcia · 05-27-2022

Hi, I wondered if some of you have had this issue before and how it can be solved. In a Databricks Job, we have a UBQ with a Painless script for ES. these are the options. Staging and prod are the same configurations, but Staging is failing with the ...

Raymond_Garcia · 11-16-2022

The error was more related to a lot of stuff on the AWS side, so we deleted and cleared the SQS and SNS. we also used the CloudFilesAWSResourceManagerval manager = CloudFilesAWSResourceManager .newManager .option("path", filePath) .create...

Raymond_Garcia · 10-26-2022

it seems it was a bug with datarbricks, but it affected prod

Raymond_Garcia · 10-17-2022

Hello user 611969 :), yes they are present in the repo

Raymond_Garcia · 07-06-2022

SELECT count(*)FROM ( SELECT explode(mmycolumn) FROM table WHERE created_year = 2022 and created_month = 7 and created_day = 5)WHERE col.field is not null and col.field.field! = "signal"

Raymond_Garcia · 06-29-2022

Hello, I was able to reduce the time significantly. I used the OPTIMIZE keyword before starting processing.Thanks!

Databricks

User Stats

User Activity

Problem with Autoloader, S3, and wildcard

Issue with databricks and DRIVER_LIBRARY_INSTALLATION_FAILURE.?

query array in SQL

Databricks Job is slower.

EsHadoopNoNodesLeftException: Connection error (check network and/or proxy settings)- all nodes failed; tried [[elasticsearch_server:80]]

Re: Problem with Autoloader, S3, and wildcard

Re: Issue with databricks and DRIVER_LIBRARY_INSTALLATION_FAILURE.?

Re: Issue with databricks and DRIVER_LIBRARY_INSTALLATION_FAILURE.?

Re: query array in SQL

Re: Databricks Job is slower.