cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

daz
by New Contributor III
  • 3445 Views
  • 8 replies
  • 3 kudos

DLT managed by non-existent pipeline

I am building out a new DLT pipeline and have since had to rebuild it from scratch. Having deleted the old pipeline and constructed a new one I now get this error:Table 'X' is already managed by pipeline 'Y'. As I only have the one pipeline how would...

  • 3445 Views
  • 8 replies
  • 3 kudos
Latest Reply
Vidula
Honored Contributor
  • 3 kudos

Hi there @Darron Smith​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you...

  • 3 kudos
7 More Replies
rubenesanchez
by New Contributor II
  • 2746 Views
  • 4 replies
  • 0 kudos

How dynamically pass a string parameter to a Delta Live Table Pipeline when calling from Azure Data Factory using REST API

I want to pass some context information to the delta live tables pipeline when calling from Azure Data Factory. I know the body of the API call supports Full Refresh parameter but I wonder if I can add my own custom parameters and how this can be re...

  • 2746 Views
  • 4 replies
  • 0 kudos
Latest Reply
Manjula_Ganesap
Contributor
  • 0 kudos

@rubenesanchez  - Did you find a solution to your problem? I have the same question. 

  • 0 kudos
3 More Replies
Dave_Nithio
by Contributor
  • 4377 Views
  • 4 replies
  • 7 kudos

Resolved! Delta Live Table Pipeline with Multiple Notebooks

I have two notebooks created for my Delta Live Table pipeline. The first is a utils notebook with functions I will be reusing for other pipelines. The second contains my actual creation of the delta live tables. I added both notebooks to the pipeline...

image.png
  • 4377 Views
  • 4 replies
  • 7 kudos
Latest Reply
fecavalc08
New Contributor III
  • 7 kudos

Hi @Vivian Wilfred​ and @Dave Wilson​ we solved our reusability code with repos and pointing the code to our main code:sys.path.append(os.path.abspath('/Workspace/Repos/[your repo]/[folder with the python scripts'))from your_class import *It just wor...

  • 7 kudos
3 More Replies
Aaron1234567
by New Contributor III
  • 1935 Views
  • 9 replies
  • 3 kudos

Delta Live Table Pipeline - Azure cluster fail

Failed to launch pipeline cluster 0802-171503-4m02lexd: The operation could not be performed on your account with the following error message: azure_error_code: OperationNotAllowed, azure_error_message: Operation could not be completed as it results ...

  • 1935 Views
  • 9 replies
  • 3 kudos
Latest Reply
Vidula
Honored Contributor
  • 3 kudos

Hi there @Aaron LeBato​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you...

  • 3 kudos
8 More Replies
abhay_sud
by New Contributor II
  • 2594 Views
  • 5 replies
  • 3 kudos

Delta Live Table Pipeline Load Error

I am getting this error when running Delta Live Table Pipelines pulling from a source in our sandbox's local dbfs folder. It says the user is not authorized to perform this operation, whereas I am able to see the data when I run a simple select state...

image image image
  • 2594 Views
  • 5 replies
  • 3 kudos
Latest Reply
Anonymous
Not applicable
  • 3 kudos

Hey there @Abhay Sudhakaran​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution and mark an answer as best? Else please let us know if you need more help. We'd love to hear fr...

  • 3 kudos
4 More Replies
Anand_Ladda
by Honored Contributor II
  • 589 Views
  • 1 replies
  • 0 kudos
  • 589 Views
  • 1 replies
  • 0 kudos
Latest Reply
Anand_Ladda
Honored Contributor II
  • 0 kudos

DLT Pipeline results are published to the "Storage Location" defined as part of configuring the Pipeline. Ex:- https://docs.databricks.com/_images/dlt-create-notebook-pipeline.pngIf an explicit Storage Location is not specified, the pipeline results ...

  • 0 kudos
Labels