New to PySpark

[Anku_]

Hi all,

I am trying to get the domain from an email field using below expression; but getting an error.

Kindly help.

df.select(df.email, substring(df.email,instr(df.email,'@'),length(df.email).alias('domain')))

[Anku_]

Note: I have also tried:

 df.select(df.email, substring(df.email,instr(df.email,'@')+1,length(df.email) - instr(df.email,'@')+1).alias('domain'))

[Walter_C]

In your case, you want to extract the domain from the email, which starts from the position just after '@'. So, you should add 1 to the position of '@'. Also, the length of the substring should be the difference between the total length of the email and the position of '@'.

Can you try with the below:

from pyspark.sql.functions import instr, length, substring

df.select(df.email, substring(df.email, instr(df.email, '@') + 1, length(df.email) - instr(df.email, '@')).alias('domain'))

This code will create a new column 'domain' that contains the domain of the email.