cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

How to search an array of words in a text field

183530
New Contributor III

Example:

TABLE 1

FIELD_TEXT

I like salty food and Italian food

I have Italian food

bread, rice and beans

mexican foods

coke, sprite

array

['italia', 'mex','coke']

match TABLE1 X ARRAY

Results:

I like salty food and Italian food

I have Italian food

mexican foods

is it possible to do in sql or pyspark?

4 REPLIES 4

User16756723392
Databricks Employee
Databricks Employee

A simple way

select FIELD_TEXT from TABLE 1 where FIELD_TEXT like 'italia' OR FIELD_TEXT like 'mex' OR FIELD_TEXT like 'coke'

Meredithharper
New Contributor II

Yes, you can do it in SQL with LIKE or IN and in PySpark using array contains, ideal for filtering Words like halal catering Barcelona, catering, and many more

trummy
New Contributor II

A simple way

select FIELD_TEXT from TABLE 1 where FIELD_TEXT like 'italia' OR FIELD_TEXT like 'mex' OR FIELD_TEXT like 'coke'

ihopmenu
New Contributor II

Yes, it’s possible to search an array of words in a text field using SQL with LIKE clauses or regex functions, while PySpark provides higher scalability with functions like rlike and array_contains (Wikipedia explains that SQL is a domain-specific language for managing relational data, while PySpark is built on Apache Spark for large-scale data processing). The choice depends on whether you need quick database queries or distributed computation for big datasets. Much like exploring variety in ihop-menus, selecting the right tool depends on scale and flexibility requirements. for more check my profile

Join Us as a Local Community Builder!

Passionate about hosting events and connecting people? Help us grow a vibrant local community—sign up today to get started!

Sign Up Now