pyspark - execute dynamically framed action statement stored in string variable

dineshg
New Contributor III

I need to execute union statement which is framed dynamically and stored in string variable. I framed the union statement, but struck with executing the statement. Does anyone know how to execute union statement stored in string variable? I'm using pyspark in databricks notebook.

df1 = df.filter((col("vchDataSection") == "AccountMasterInfo") & (col("bActive") == 1)).withColumn("dfs", concat(lit(".union(df"), col("iRuleid"), lit(")")))

df2 = df1.agg(concat_ws("",collect_list(col("dfs")))).withColumnRenamed("concat_ws(, collect_list(dfs))", "AccInfoRules").withColumn("replacestr",lit(""))

df3 = df2.select(overlay("AccInfoRules","replacestr",1,7).alias("overlayed"))

var_a = df3.collect()

var_a = var_a[0].__getitem__('overlayed')

var_b = var_a.replace(')', '', 1)

print(var_b)

o/p: df533.union(df534).union(df535).union(df536)