topic Re: Is there a function similar to split_part, json_extract_scalar? in Data Engineering

Is there a function similar to split_part, json_extract_scalar?

dulu — Sat, 10 Dec 2022 03:38:27 GMT

I am using spark_sql version 3.2.1. Is there a function that can replace

split_part,

json_extract_scalar

are not?

Re: Is there a function similar to split_part, json_extract_scalar?

Ankush — Sat, 10 Dec 2022 11:55:00 GMT

pyspark.sql.functions.

get_json_object

(col, path)[source]

Extracts json object from a json string based on json path specified, and returns json string of the extracted json object. It will return null if the input json string is invalid.

Re: Is there a function similar to split_part, json_extract_scalar?

dulu — Fri, 16 Dec 2022 04:52:31 GMT

Thanks @Ankush Jadhao for answer, I tried but only returns the first result in the cell. They seem to only take the first character with the correct result. I need out put like the table below.

I have the following input:

I am looking for a way to split the characters in the item_order_detail column into 2 columns itemID and itemName. As below output table uses SQL function in databricks with spark_sql version 3.2.1.