I have an issue where Spark in not submiting any task, on any worksapce or cluster, even SQLWarehouse.Even for very simple code it hangs forever.Anyone ever faced something similar? Our infra is AWS.
Found the solution: Turned out to be an issue with the Security Groups. The internal security group communication was not open to all ports for TCP and UDP. After fixing that the jobs ran fine. Seems like we did require more workers too.
Hi Kaniz, thanks for the reply.I went thought the log and I see this:[0;31mKeyboardInterrupt[0m:23/10/26 21:06:04 INFO ProgressReporter$: Removed result fetcher for 7389618138579564799_6933402728921115182_ee7173b16c654fea9ca6968ef33e553023/10/26 21:0...
