cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

@dlt.table throws error AttributeError: module 'dlt' has no attribute 'table'

DE_K
New Contributor II

Hi Everyone,

I am new to DLT and am trying to run below code to create table dynamically. But I get error "AttributeError: module 'dlt' has no attribute 'table'". 

code snippet:

def generate_tables(model_name😞
  try:
    spark.sql("select * from dlt.{0}".format(model_name))
    @dlt.append_flow(target = model_name)
    def append_raw():
      return(spark.table("dlt.{0}".format(model_name)).where("source_datetime>=''"))
  except:
    @dlt.table(
    name=model_name,
    comment="RAW: {}".format(model_name)
    )  
    def create_table():    
      tableSchema=createSchema('Meta_Data.csv',model_name)
      return(spark.read.format("csv").option("delimiter","|").option("header","true").option("escape","\"").load("{0}/{1}".format(<path to file>),schema=tableSchema).withColumn("source_datetime",current_timestamp()))
 
import dlt
generate_tables('dlt_Raw')
 
when I run the generate_tables function with parameter, I get the above error.
DE_K_0-1712148007690.png

I have the dlt module loaded via pip install

4 REPLIES 4

YuliyanBogdanov
New Contributor III

Hey @DE_K , It's hard to follow without some more context, but shouldn't that be live instead of dlt - "select * from live.{0}".format(model_name)", considering you have already created the table.

Hi @YuliyanBogdanov , That piece of code is under @dlt.append_flow which I believe would append the data, the exception section would first create the table. The code follows same path and goes to exception section but throws error at @dlt.table . So if you have any idea as to why its happening, please let me know.

YuliyanBogdanov
New Contributor III

Thank You, @DE_K. I see your point. I believe you are using the @dlt.table instead of @dlt.create_table to begin with, since want the table to be created and not define and existing one. (https://community.databricks.com/t5/data-engineering/difference-dlt-table-and-dlt-create-table-decor...). Let me know if this helped.

 

Hi @YuliyanBogdanov , no luck with that as well. getting same error. Not sure what is it that I am missing.

DE_K_0-1712310778914.png

I installed dlt using pip install dlt. I believe that is the way to go.

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group