This is an elegant one, it uses isDeltaTable() from DeltaTableUtils and dbutils.

import org.apache.spark.sql.delta.DeltaTableUtils
 
val s3Path = "s3://my-bucket/my-folder"
 
// Get a list of all the folders in the S3 path
val folders = dbutils.fs.ls(s3Path).map(_.path)
 
// Filter out any folders that are Delta tables
val nonDeltaFolders = folders.filter(path => !DeltaTableUtils.isDeltaTable(path))
 
// Print the resulting list of folders
nonDeltaFolders.foreach(println)

Note: this is also available in python (package DeltaTableUtils is DeltaTable)

Thanks & Regards,

Nandini