https://community.databricks.com/t5/data-engineering/partition-in-spark-with-subqeury-which-include-union/m-p/81859#M36433 <P>I have a SQL query like this:<BR />select ... from table1 where id in (slect id from table 1 where (some condition) UNION select id from table2 where (some condition)) table1<BR /><BR />I have made a partition of 200 where upper bound is 200 and lower bound is 0 and partition will be done on partition_key column which has value ranged from 1 to 200. I am using JDBC connector. I am passing options("dbtable,table)&nbsp; where table is the query mentioned above.<BR /><BR />How will the internal query in Spark look like? Since we are using UNION will it not affect the partitioning?&nbsp;</P> Mon, 05 Aug 2024 10:26:13 GMT Jeewan 2024-08-05T10:26:13Z https://community.databricks.com/t5/data-engineering/partition-in-spark-with-subqeury-which-include-union/m-p/81859#M36433 <P>I have a SQL query like this:<BR />select ... from table1 where id in (slect id from table 1 where (some condition) UNION select id from table2 where (some condition)) table1<BR /><BR />I have made a partition of 200 where upper bound is 200 and lower bound is 0 and partition will be done on partition_key column which has value ranged from 1 to 200. I am using JDBC connector. I am passing options("dbtable,table)&nbsp; where table is the query mentioned above.<BR /><BR />How will the internal query in Spark look like? Since we are using UNION will it not affect the partitioning?&nbsp;</P> Mon, 05 Aug 2024 10:26:13 GMT https://community.databricks.com/t5/data-engineering/partition-in-spark-with-subqeury-which-include-union/m-p/81859#M36433 Jeewan 2024-08-05T10:26:13Z