Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
09-09-2024 08:52 PM
Hi @abueno ,
To replicate a SQL `not like '%var4%'` clause in the Dataframe API, you could use `rlike` with negation using `~` such as:
df.filter(~col('col4').rlike('var4')).display()
Here's a basic reproducible example:
df = (spark.range(10).withColumn("col4", f.lit("var3"))).union(
spark.range(10).withColumn("col4", f.lit("var4")))
df.filter(~col('col4').rlike('var4')).groupBy('col4').count().display()
col4 count
var3 10
Hope this helps.