DLT Pipeline failed to Start due to "The Execution Contained atleast one disallowed language

[Arumugam]

Hi ,

im trying to setup DLT pipeline ,its a basic pipeline for testing purpose im facing the issue while starting the pipeline , any help is appreciated

 Code :

@dlt.table(name="dlt_bronze_cisco_hardware")

def dlt_cisco_networking_bronze_hardware():

 return (

 spark.readStream.format("cloudFiles")\

   .option("cloudFiles.format", "json")\

   .option("cloudFiles.schemaLocation","abfss://adt-calfit-adls@adtedfdatalake.dfs.core.windows.net/DLT_Schema")\

   .load("abfss://adt-calfit-adls@adtedfdatalake.dfs.core.windows.net/Landing/DLT_Testing/Hardware/*/*/*/*.json")

 )

[Pat]

HI @Arumugam Ramachandran​ ,

first though - did you chose PYTHON CELL for your code?

or maybe this post will help: https://community.databricks.com/s/question/0D58Y00009DD3LZSA1/python-code-not-working-in-dbr-104-lt...

[Arumugam]

Hi Pat , Yes i have selected Python as the script for the cell

[Vivian_Wilfred]

Hi @Arumugam Ramachandran​ seems like you have a spark config set on your DLT job cluster that allows only python and SQL code. Check the spark config (cluster policy).

In any case, the python code should work. Verify the notebook's default language, it should not be scala.

Check this doc for additional details.

https://docs.databricks.com/notebooks/notebooks-manage.html#execution-contexts

Mark a comment as 'best answer' to resolve the query, thanks.

[Arumugam]

Hi vivian , the notebook has default as python , but still facing the issue

[Vivian_Wilfred]

@Arumugam Ramachandran​ Please remove "spark.databricks.repl.allowedLanguages" config for the DLT clusters if you have it in the policy. DLT only works with python and SQL notebook anyways and we don't need to explicitly set this config.