You can disable broadcast join by adding the following Spark configuration to you notebook:

spark.conf.set("spark.sql.autoBroadcastJoinThreshold", -1)

In addition, you can also add this configuration to your cluster:

spark.sql.autoBroadcastJoinThreshold -1

You can disable it at notebook level or cluster level.