DLT table not picked in python notebook

[Mathias_Peters]

Hi, 

I am a bit stumped atm bc I cannot figure out how to get a DLT table definition picked up in a Python notebook. 

1. I created a new notebook in python

2. added the following code: 

%python
import dlt
from pyspark.sql.functions import *

@dlt.table(
    comment="Some simple test",
    name="trx_dlt"
)
def transactions_live_table():
    df = spark.read.json("s3://FOLDER_LOCATION/Transaction/")
    return df

3. I created a new DLT pipeline and linked the Notebook. Linking did work bc the Notebook UI now allows me to validate my pieline code.

4. I press play and get 

   a. a warning the python magic command is not supported: 

Magic commands (e.g. %py, %sql and %run) are not supported with the exception of %pip within a Python notebook. Cells containing magic commands are ignored. Unsupported magic commands were found in the following notebooks

/Users/mathias.peters@epicompany.eu/DLT-TRX: %python

  b. an error that no DLT tables were discovered in the code: 

How does that work? Where is my mistake here?

Kind regards, 

Mathias

[Mathias_Peters]

Update: this seems to be an issue with the notebook type and the magic commands at the beginnning. I have created a new DLT pipeline without attaching a notebook to it. That triggered the creation of an empty Python notebook. Switching this to SQL added the %sql command. Running the pipeline produced the same no tables discovered error as above. What is the correct way of specifying the notebook type here? (super unintuitive btw) 

[Mathias_Peters · Accepted solution]

Ok, it seems that the default language of the notebook and the language of a particular cell can clash. If the default is set to Python, switching a cell to SQL won't work in DLT and vice versa. This is super unintuitive tbh.