cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

DLT Pipeline failed to Start due to "The Execution Contained atleast one disallowed language

Arumugam
New Contributor II

Hi ,

im trying to setup DLT pipeline ,its a basic pipeline for testing purpose im facing the issue while starting the pipeline , any help is appreciated

 Capture.PNGCaptureCode :

@dlt.table(name="dlt_bronze_cisco_hardware")

def dlt_cisco_networking_bronze_hardware():

 return (

 spark.readStream.format("cloudFiles")\

   .option("cloudFiles.format", "json")\

   .option("cloudFiles.schemaLocation","abfss://adt-calfit-adls@adtedfdatalake.dfs.core.windows.net/DLT_Schema")\

   .load("abfss://adt-calfit-adls@adtedfdatalake.dfs.core.windows.net/Landing/DLT_Testing/Hardware/*/*/*/*.json")

 )

5 REPLIES 5

Pat
Honored Contributor III

HI @Arumugam Ramachandranโ€‹ ,

first though - did you chose PYTHON CELL for your code?

or maybe this post will help: https://community.databricks.com/s/question/0D58Y00009DD3LZSA1/python-code-not-working-in-dbr-104-lt...

Arumugam
New Contributor II

Hi Pat , Yes i have selected Python as the script for the cell

Vivian_Wilfred
Honored Contributor
Honored Contributor

Hi @Arumugam Ramachandranโ€‹ seems like you have a spark config set on your DLT job cluster that allows only python and SQL code. Check the spark config (cluster policy).

In any case, the python code should work. Verify the notebook's default language, it should not be scala.

imageCheck this doc for additional details.

https://docs.databricks.com/notebooks/notebooks-manage.html#execution-contexts

Mark a comment as 'best answer' to resolve the query, thanks.

Hi vivian , the notebook has default as python , but still facing the issue

@Arumugam Ramachandranโ€‹ Please remove "spark.databricks.repl.allowedLanguages" config for the DLT clusters if you have it in the policy. DLT only works with python and SQL notebook anyways and we don't need to explicitly set this config.

Join 100K+ Data Experts: Register Now & Grow with Us!

Excited to expand your horizons with us? Click here to Register and begin your journey to success!

Already a member? Login and join your local regional user group! If there isn’t one near you, fill out this form and we’ll create one for you to join!