How execute SET spark.sql.sources.partitionOverwri...

RGSLCA · ‎06-05-2026

Hi,

I am able to execute the INSERT OVERWRITE TABLE <tables> PARTITION command , in a notebook cell

SET spark.sql.sources.partitionOverwriteMode=dynamic;

DECLARE OR REPLACE VARIABLE v_load_date DATE;

SET VAR v_load_date = DATE '2026-05-03';

INSERT OVERWRITE TABLE sil_test_rg

PARTITION (bed)

SELECT *

FROM brz_test_rg

WHERE bed IN (

SELECT DISTINCT bed

FROM brz_test_rg

WHERE load_date = v_load_date

)

and load_date = v_load_date;

Once I converted the code to a stored procedure, I dont have the REPLACE functionality,

I tried wrapping the command "SET spark.sql.sources.partitionOverwriteMode = dynamic" in a execute immediate statement, but it doesn't help.

BEGIN

DECLARE slv_ingest_dt DATE;

SET slv_ingest_dt = cast('2026-05-03' as date);

execute immediate 'SET spark.sql.sources.partitionOverwriteMode=dynamic';

CALL ingest_silver_table('sil_test_rg','brz_test_rg',slv_ingest_dt);

END;

How do I use the SELECTIVE OVERWRITE option in stored procedures ?

How execute SET spark.sql.sources.partitionOverwriteMode = dynamic; in SQL Stored procedures