You can run this query to get the long running queries and then kill the query you wanted to kill.

# Step 1: Get active queries
active_queries = spark.sql("SHOW PROCESSLIST")
active_queries.show(truncate=False)

# Step 2: Identify the query ID you want to kill
# (Assume you noted the ID as '12345')

# Step 3: Kill the query
spark.sql("KILL QUERY 12345")

@slakshmanan