cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
cancel
Showing results for 
Search instead for 
Did you mean: 

@dlt.table throws error AttributeError: module 'dlt' has no attribute 'table'

DE_K
New Contributor II

Hi Everyone,

I am new to DLT and am trying to run below code to create table dynamically. But I get error "AttributeError: module 'dlt' has no attribute 'table'". 

code snippet:

def generate_tables(model_name😞
  try:
    spark.sql("select * from dlt.{0}".format(model_name))
    @dlt.append_flow(target = model_name)
    def append_raw():
      return(spark.table("dlt.{0}".format(model_name)).where("source_datetime>=''"))
  except:
    @dlt.table(
    name=model_name,
    comment="RAW: {}".format(model_name)
    )  
    def create_table():    
      tableSchema=createSchema('Meta_Data.csv',model_name)
      return(spark.read.format("csv").option("delimiter","|").option("header","true").option("escape","\"").load("{0}/{1}".format(<path to file>),schema=tableSchema).withColumn("source_datetime",current_timestamp()))
 
import dlt
generate_tables('dlt_Raw')
 
when I run the generate_tables function with parameter, I get the above error.
DE_K_0-1712148007690.png

I have the dlt module loaded via pip install

4 REPLIES 4

YuliyanBogdanov
New Contributor II

Hey @DE_K , It's hard to follow without some more context, but shouldn't that be live instead of dlt - "select * from live.{0}".format(model_name)", considering you have already created the table.

Hi @YuliyanBogdanov , That piece of code is under @dlt.append_flow which I believe would append the data, the exception section would first create the table. The code follows same path and goes to exception section but throws error at @dlt.table . So if you have any idea as to why its happening, please let me know.

YuliyanBogdanov
New Contributor II

Thank You, @DE_K. I see your point. I believe you are using the @dlt.table instead of @dlt.create_table to begin with, since want the table to be created and not define and existing one. (https://community.databricks.com/t5/data-engineering/difference-dlt-table-and-dlt-create-table-decor...). Let me know if this helped.

 

Hi @YuliyanBogdanov , no luck with that as well. getting same error. Not sure what is it that I am missing.

DE_K_0-1712310778914.png

I installed dlt using pip install dlt. I believe that is the way to go.

Welcome to Databricks Community: Lets learn, network and celebrate together

Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections. 

Click here to register and join today! 

Engage in exciting technical discussions, join a group with your peers and meet our Featured Members.