Hello community:

I don´t know why my Delta Live table workflow fails into this step. This is the configuration I have for the pipeline:

{
"id": "**",
"pipeline_type": "WORKSPACE",
"clusters": [
{
"label": "default",
"spark_conf": {***},
"num_workers": 3
},
{
"label": "maintenance",
"spark_conf": {****},
"num_workers": 3
}
],
"development": false,
"continuous": false,
"channel": "CURRENT",
"photon": false,
"libraries": [
{
"notebook": {
"path": "/GeneracionAutomatica/DLT Generator (bronze)"
}
},
{
"notebook": {
"path": "/GeneracionAutomatica/DLT Generator (staging)"
}
}
],
"name": "Delta Live Tables (bronze)",
"edition": "ADVANCED",
"storage": "****",
"configuration": {
"pipelines.autoOptimize.zOrderCols": "\"timestamp\"",
"pipelines.numUpdateRetryAttempts": "7"
},
"target": "myrl"
}

Note: *** is information confidential.

The result sometimes is this:

But sometimes it works and complete de pipeline!!

Anyone has any idea? Thanks!!