cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Data Engineering
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

DLT Pipeline failed to Start due to "The Execution Contained atleast one disallowed language

Arumugam
New Contributor II

Hi ,

im trying to setup DLT pipeline ,its a basic pipeline for testing purpose im facing the issue while starting the pipeline , any help is appreciated

 Capture.PNGCaptureCode :

@dlt.table(name="dlt_bronze_cisco_hardware")

def dlt_cisco_networking_bronze_hardware():

 return (

 spark.readStream.format("cloudFiles")\

   .option("cloudFiles.format", "json")\

   .option("cloudFiles.schemaLocation","abfss://adt-calfit-adls@adtedfdatalake.dfs.core.windows.net/DLT_Schema")\

   .load("abfss://adt-calfit-adls@adtedfdatalake.dfs.core.windows.net/Landing/DLT_Testing/Hardware/*/*/*/*.json")

 )

5 REPLIES 5

Pat
Honored Contributor III

HI @Arumugam Ramachandranโ€‹ ,

first though - did you chose PYTHON CELL for your code?

or maybe this post will help: https://community.databricks.com/s/question/0D58Y00009DD3LZSA1/python-code-not-working-in-dbr-104-lt...

Arumugam
New Contributor II

Hi Pat , Yes i have selected Python as the script for the cell

Vivian_Wilfred
Honored Contributor
Honored Contributor

Hi @Arumugam Ramachandranโ€‹ seems like you have a spark config set on your DLT job cluster that allows only python and SQL code. Check the spark config (cluster policy).

In any case, the python code should work. Verify the notebook's default language, it should not be scala.

imageCheck this doc for additional details.

https://docs.databricks.com/notebooks/notebooks-manage.html#execution-contexts

Mark a comment as 'best answer' to resolve the query, thanks.

Hi vivian , the notebook has default as python , but still facing the issue

@Arumugam Ramachandranโ€‹ Please remove "spark.databricks.repl.allowedLanguages" config for the DLT clusters if you have it in the policy. DLT only works with python and SQL notebook anyways and we don't need to explicitly set this config.

Welcome to Databricks Community: Lets learn, network and celebrate together

Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections. 

Click here to register and join today! 

Engage in exciting technical discussions, join a group with your peers and meet our Featured Members.