Databricks

Ancil · 03-23-2023

Hi Team, I have created new databricks workspace for using private network. I want to read/write/update delta table created in old databricks and stored in ADLS as delta.Any one please help me on this?.....I tried to create delta table in same loca...

Ancil · 03-02-2023

I am getting below error some time run my databricks notebook from ADF, If the executor node is one then it works fine, if it increases 2 or more some times its failing on same data.Cluster Detail : Standard_F4s_v2 · Workers: Standard_F4s_v2 · 1-8 wo...

Ancil · 01-18-2023

I have pandas_udf, its working for 4 rows, but I tried with more than 4 rows getting below error.PythonException: 'RuntimeError: The length of output in Scalar iterator pandas UDF should be the same with the input's; however, the length of output was...

Ancil · 01-17-2023

I have pandas_udf, its working for 1 rows, but I tried with more than one rows getting below error.PythonException: 'RuntimeError: The length of output in Scalar iterator pandas UDF should be the same with the input's; however, the length of output w...

Ancil · 12-01-2022

Scenario: I Have a dataframe with more than 1000 rows, each row having a file path and result data column. I need to loop through each row and write files to the file path, with data from the result column.what is the easiest and time effective way ...

Ancil · 05-09-2023

Hi @Swetha Nandajan Please find the full error log. I have job running every one hour my notebook worked for 20 runs. after that am getting below error. Am creating new job cluster for every run from ADF. Cluster details : Driver: Standard_F32s_v2...

Ancil · 04-10-2023

I have pasted all the logs available in data bricks

Ancil · 03-24-2023

Thank you for update, Is there any way to convert the table to DELTA tables.And use the same data and ADLS location new databricks workspace

Ancil · 03-23-2023

Is there any way to use same delta table adls location in new Databricks workspace . Because I need old data in new Databricks workspace

Ancil · 03-23-2023

I have created new databricks workspace for using private network. I want to read/write/update delta table created in old databricks and stored in ADLS as delta.Any one please help me on this?.....I tried to create delta table in same location but ge...

Databricks

User Stats

User Activity

Cannot create table The associated location ('dbfs:/mnt/[REDACTED]/folder/table_location') is not empty but it's not a Delta table

Job aborted due to stage failure: Task 1863 in stage 10.0 failed 4 times, most recent failure: Lost task 1863.3 in stage 10.0 (TID 2021) (10.0.4.7 executor 2): org.apache.spark.SparkException: Python worker exited unexpectedly (crashed): Fatal Python erro

PythonException: 'RuntimeError: The length of output in Scalar iterator pandas UDF should be the same with the input's; however, the length of output was 1 and the length of input was 2.'.

PythonException: 'RuntimeError: The length of output in Scalar iterator pandas UDF should be the same with the input's; however, the length of output was 1 and the length of input was 2.'.

Any on please suggest how we can effectively loop through PySpark Dataframe .

Re: Job aborted due to stage failure: Task 1863 in stage 10.0 failed 4 times, most recent failure: Lost task 1863.3 in stage 10.0 (TID 2021) (10.0.4.7 executor 2): org.apache.spark.SparkException: Python worker exited unexpectedly (crashed): Fatal Python erro

Re: Job aborted due to stage failure: Task 1863 in stage 10.0 failed 4 times, most recent failure: Lost task 1863.3 in stage 10.0 (TID 2021) (10.0.4.7 executor 2): org.apache.spark.SparkException: Python worker exited unexpectedly (crashed): Fatal Python erro

Re: Cannot create table The associated location ('dbfs:/mnt/[REDACTED]/folder/table_location') is not empty but it's not a Delta table

Re: Cannot create table The associated location ('dbfs:/mnt/[REDACTED]/folder/table_location') is not empty but it's not a Delta table

Re: Cannot create table The associated location ('dbfs:/mnt/[REDACTED]/folder/table_location') is not empty but it's not a Delta table