DevOps Fundamentals: 11 - Performing Integration Tests: Replace bronze with silver

[Daniel_B]

ingest-bronze-silver_sdp.py

Change {"quality": "bronze"} to {"quality": "silver"}

####################################################################################

## B. Silver Table ##

####################################################################################

@dp.table(

comment = "This table will create, drop and categorize columns from the bronze table.",

table_properties = {"quality": "bronze"}

)

def health_silver():

return (

 dp

 .read_stream("health_bronze")

 .withColumn("HighCholest_Group", project_functions.high_cholest_map("HighCholest")) # UDF - highcholest_map 

 .withColumn("Age_Group", project_functions.group_ages_map("Age")) # UDF - group_ages_map Age

 .drop("file_name", "file_modification_time", "processing_time") # Drop unnecessary metadata columns

 )

[drist029ity]

Hello, Just update table_properties={"quality": "bronze"} to table_properties={"quality": "silver"} in the health_silver() table definition.  

[Aravind_Reddy]

Hello @Daniel_B 

Good catch! The health_silver() definition should use table_properties={"quality": "silver"} to keep the table metadata consistent with the intended medallion layer.

It would also be useful to include an integration-test assertion that validates the expected table properties and confirms that the Silver transformations and metadata-column removals work as intended. This could help catch similar configuration mistakes before deployment.