topic Re: DLT Pipeline failed to Start due to "The Execution Contained atleast one disallowed language in Data Engineering

DLT Pipeline failed to Start due to "The Execution Contained atleast one disallowed language

Arumugam — Thu, 17 Nov 2022 11:12:07 GMT

Hi ,

im trying to setup DLT pipeline ,its a basic pipeline for testing purpose im facing the issue while starting the pipeline , any help is appreciated

Code :

@dlt.table(name="dlt_bronze_cisco_hardware")

def dlt_cisco_networking_bronze_hardware():

return (

spark.readStream.format("cloudFiles")\

.option("cloudFiles.format", "json")\

.option("cloudFiles.schemaLocation","abfss://adt-calfit-adls@adtedfdatalake.dfs.core.windows.net/DLT_Schema")\

.load("abfss://adt-calfit-adls@adtedfdatalake.dfs.core.windows.net/Landing/DLT_Testing/Hardware/*/*/*/*.json")

)

Re: DLT Pipeline failed to Start due to "The Execution Contained atleast one disallowed language

Pat — Thu, 17 Nov 2022 12:47:23 GMT

HI @Arumugam Ramachandran ,

first though - did you chose PYTHON CELL for your code?

or maybe this post will help: https://community.databricks.com/s/question/0D58Y00009DD3LZSA1/python-code-not-working-in-dbr-104-lts-shared-mode

Re: DLT Pipeline failed to Start due to "The Execution Contained atleast one disallowed language

Arumugam — Thu, 17 Nov 2022 13:23:26 GMT

Hi Pat , Yes i have selected Python as the script for the cell

Re: DLT Pipeline failed to Start due to "The Execution Contained atleast one disallowed language

Vivian_Wilfred — Thu, 17 Nov 2022 13:30:45 GMT

Hi @Arumugam Ramachandran seems like you have a spark config set on your DLT job cluster that allows only python and SQL code. Check the spark config (cluster policy).

In any case, the python code should work. Verify the notebook's default language, it should not be scala.

Check this doc for additional details.

https://docs.databricks.com/notebooks/notebooks-manage.html#execution-contexts

Mark a comment as 'best answer' to resolve the query, thanks.

Re: DLT Pipeline failed to Start due to "The Execution Contained atleast one disallowed language

Arumugam — Thu, 17 Nov 2022 14:40:42 GMT

Hi vivian , the notebook has default as python , but still facing the issue

Re: DLT Pipeline failed to Start due to "The Execution Contained atleast one disallowed language

Vivian_Wilfred — Thu, 17 Nov 2022 15:04:45 GMT

@Arumugam Ramachandran Please remove "spark.databricks.repl.allowedLanguages" config for the DLT clusters if you have it in the policy. DLT only works with python and SQL notebook anyways and we don't need to explicitly set this config.