I would

• remove maxOffsetsPerTrigger,
• analyze spark UI
• use the display() to see how the stream is read (without writing it to delta) or alternatively write to noop as problem can be with read or write or with kafka itself
• set for stream shuffle partitions to the number of cores

spark.conf.set("spark.sql.shuffle.partitions", sc.DefaultParallelism)

in stream add .option("minPartitions", sc.DefaultParallelism)