you can use for loop, it will process parallelly based on the cluster size.Define the dlt logic in a function,def dlt_logic(table_name): ........ then pass your table names in a list to the function. table_names = ["table1", "table2", "table3"]f...
can we pass the /api/2.0/pipelines/{pipeline_id} url to start the dlt pipeline inside a asynchronous function,this is for parallelly executing multiple tables.
can we run a dlt pipeline multiple time at the same time using different parameters using rest api call with asyncio.i have created a function to start the pipeline using rest api.when calling the function with asyncio , i am getting [409 Conflict]> ...
if we use for loop to pass table names, it will be handled one by one, right?if yes, can you suggest any other methods like I need to process 'n' number of tables at a time .
